TM

Apache Solr Reference Guide Covering Apache Solr 6.3

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY

KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Apache and the Apache feather logo are trademarks of The Apache Software Foundation. Apache Lucene, Apache Solr and their respective logos are trademarks of the Apache Software Foundation. Please see the Apache Trademark Policy for more information.

Fonts used in the Apache Solr Reference Guide include Raleway, licensed under the SIL Open Font License, 1.1.

Apache Solr Reference Guide‌

This reference guide describes Apache Solr, the open source solution for search. You can download Apache Solr from the Solr website at http://lucene.apache.org/solr/.

This Guide contains the following sections:

Getting Started: This section guides you through the installation and setup of Solr.

Using the Solr Administration User Interface: This section introduces the Solr Web-based user interface. From your browser you can view configuration files, submit queries, view logfile settings and Java environment settings, and monitor and control distributed configurations.

Documents, Fields, and Schema Design: This section describes how Solr organizes its data for indexing. It explains how a Solr schema defines the fields and field types which Solr uses to organize data within the document files it indexes.

Understanding Analyzers, Tokenizers, and Filters: This section explains how Solr prepares text for indexing and searching. Analyzers parse text and produce a stream of tokens, lexical units used for indexing and searching. Tokenizers break field data down into tokens. Filters perform other transformational or selective work on token streams.

Indexing and Basic Data Operations: This section describes the indexing process and basic index operations, such as commit, optimize, and rollback.

Searching: This section presents an overview of the search process in Solr. It describes the main components used in searches, including request handlers, query parsers, and response writers. It lists the query parameters that can be passed to Solr, and it describes features such as boosting and faceting, which can be used to

fine-tune search results.

The Well-Configured Solr Instance: This section discusses performance tuning for Solr. It begins with an overview of the solrconfig.xml file, then tells you how to configure cores with solr.xml, how to configure the Lucene index writer, and more.

Managing Solr: This section discusses important topics for running and monitoring Solr. Other topics include how to back up a Solr instance, and how to run Solr with Java Management Extensions (JMX).

SolrCloud: This section describes the newest and most exciting of Solr's new features, SolrCloud, which provides comprehensive distributed capabilities.

Legacy Scaling and Distribution: This section tells you how to grow a Solr distribution by dividing a large index into sections called shards, which are then distributed across multiple servers, or by replicating a single index across multiple services.

Client APIs: This section tells you how to access Solr through various client APIs, including JavaScript, JSON, and Ruby.

About This Guide

This guide describes all of the important features and functions of Apache Solr. It is free to download from http://l ucene.apache.org/solr/.

Designed to provide high-level documentation, this guide is intended to be more encyclopedic and less of a cookbook. It is structured to address a broad spectrum of needs, ranging from new developers getting started to well-experienced developers extending their application or troubleshooting. It will be of use at any point in the application life cycle, for whenever you need authoritative information about Solr.

The material as presented assumes that you are familiar with some basic search concepts and that you can read XML. It does not assume that you are a Java programmer, although knowledge of Java is helpful when working directly with Lucene or when developing custom extensions to a Lucene/Solr installation.

Special Inline Notes

Special notes are included throughout these pages.

Hosts and Port Examples

The default port when running Solr is 8983. The samples, URLs and screenshots in this guide may show different ports, because the port number that Solr uses is configurable. If you have not customized your installation of Solr, please make sure that you use port 8983 when following the examples, or configure your own installation to use the port numbers shown in the examples. For information about configuring port numbers, see Managing Solr.

Similarly, URL examples use 'localhost' throughout; if you are accessing Solr from a location remote to the server hosting Solr, replace 'localhost' with the proper domain or IP where Solr is running.

Paths

Path information is given relative to solr.home, which is the location under the main Solr installation where Solr's collections and their conf and data directories are stored. When running the various examples

mentioned through out this tutorial (i.e., bin/solr -e techproducts) the solr.home will be a sub directory of example/ created for you automatically.‌

Getting Started

Solr makes it easy for programmers to develop sophisticated, high-performance search applications with advanced features such as faceting (arranging search results in columns with numerical counts of key terms). Solr builds on another open source search technology: Lucene, a Java library that provides indexing and search technology, as well as spellchecking, hit highlighting and advanced analysis/tokenization capabilities. Both Solr and Lucene are managed by the Apache Software Foundation (www.apache.org).

The Lucene search library currently ranks among the top 15 open source projects and is one of the top 5 Apache projects, with installations at over 4,000 companies. Lucene/Solr downloads have grown nearly ten times over the past three years, with a current run-rate of over 6,000 downloads a day. The Solr search server, which provides application builders a ready-to-use search platform on top of the Lucene search library, is the fastest growing Lucene sub-project. Apache Lucene/Solr offers an attractive alternative to the proprietary licensed search and discovery software vendors.

This section helps you get Solr up and running quickly, and introduces you to the basic Solr architecture and features. It covers the following topics:

Installing Solr: A walkthrough of the Solr installation process.

Running Solr: An introduction to running Solr. Includes information on starting up the servers, adding documents, and running queries.

A Quick Overview: A high-level overview of how Solr works.

A Step Closer: An introduction to Solr's home directory and configuration options.

Solr Start Script Reference: a complete reference of all of the commands and options available with the bin/solr script.

Solr includes a Quick Start tutorial which will be helpful if you are just starting out with Solr. You can find it online at http://lucene.apache.org/solr/quickstart.html, or in your Solr installation at $SOLR_INSTALL_D IR/docs/quickstart.html.

Installing Solr‌

This section describes how to install Solr. You can install Solr in any system where a suitable Java Runtime Environment (JRE) is available, as detailed below. Currently this includes Linux, OS X, and Microsoft Windows. The instructions in this section should work for any platform, with a few exceptions for Windows as noted.

Got Java?

You will need the Java Runtime Environment (JRE) version 1.8 or higher. At a command line, check your Java version like this:

$ java -version

java version "1.8.0_60"

Java(TM) SE Runtime Environment (build 1.8.0_60-b27)

Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

The exact output will vary, but you need to make sure you meet the minimum version requirement. We also recommend choosing a version that is not end-of-life from its vendor. If you don't have the required version, or if the java command is not found, download and install the latest version from Oracle at http://www.oracle.com/tec hnetwork/java/javase/downloads/index.html.

Installing Solr

Solr is available from the Solr website at http://lucene.apache.org/solr/.

For Linux/Unix/OSX systems, download the .tgz file. For Microsoft Windows systems, download the .zip file. When getting started, all you need to do is extract the Solr distribution archive to a directory of your choosing. When you're ready to setup Solr for a production environment, please refer to the instructions provided on the Ta king Solr to Production page. To keep things simple for now, extract the Solr distribution archive to your local home directory, for instance on Linux, do:

$ cd ~/

$ tar zxf solr-x.y.z.tgz

Once extracted, you are now ready to run Solr using the instructions provided in the Running Solr section.

Running Solr‌

This section describes how to run Solr with an example schema, how to add documents, and how to run queries.

Start the Server

If you didn't start Solr after installing it, you can start it by running bin/solr from the Solr directory.

$ bin/solr start

If you are running Windows, you can start Solr by running bin\solr.cmd instead.

bin\solr.cmd start

This will start Solr in the background, listening on port 8983.

When you start Solr in the background, the script will wait to make sure Solr starts correctly before returning to the command line prompt.

The bin/solr and bin\solr.cmd scripts allow you to customize how you start Solr. Let's work through a few examples of using the bin/solr script (if you're running Solr on Windows, the bin\solr.cmd works the same as what is shown in the examples below):

Solr Script Options

The bin/solr script has several options.

Script Help

To see how to use the bin/solr script, execute:

$ bin/solr -help

For specific usage instructions for the start command, do:

$ bin/solr start -help

Start Solr in the Foreground

Since Solr is a server, it is more common to run it in the background, especially on Unix/Linux. However, to start Solr in the foreground, simply do:

$ bin/solr start -f

If you are running Windows, you can run:

bin\solr.cmd start -f

Start Solr with a Different Port

To change the port Solr listens on, you can use the -p parameter when starting, such as:

$ bin/solr start -p 8984

Stop Solr

When running Solr in the foreground (using -f), then you can stop it using Ctrl-c. However, when running in the background, you should use the stop command, such as:

$ bin/solr stop -p 8983

The stop command requires you to specify the port Solr is listening on or you can use the -all parameter to stop all running Solr instances.

Start Solr with a Specific Bundled Example

Solr also provides a number of useful examples to help you learn about key features. You can launch the examples using the -e flag. For instance, to launch the "techproducts" example, you would do:

$ bin/solr -e techproducts

Currently, the available examples you can run are: techproducts, dih, schemaless, and cloud. See the section Ru nning with Example Configurations for details on each example.

Getting Started with SolrCloud

Running the cloud example starts Solr in SolrCloud mode. For more information on starting Solr in cloud mode, see the section Getting Started with SolrCloud.

Check if Solr is Running

If you're not sure if Solr is running locally, you can use the status command:

$ bin/solr status

This will search for running Solr instances on your computer and then gather basic information about them, such as the version and memory usage.

That's it! Solr is running. If you need convincing, use a Web browser to see the Admin Console.

http://localhost:8983/solr/

The Solr Admin interface.

If Solr is not running, your browser will complain that it cannot connect to the server. Check your port number and try again.

Create a Core

If you did not start Solr with an example configuration, you would need to create a core in order to be able to index and search. You can do so by running:

$ bin/solr create -c <name>

This will create a core that uses a data-driven schema which tries to guess the correct field type when you add documents to the index.

To see all available options for creating a new core, execute:

$ bin/solr create -help

Add Documents

Solr is built to find documents that match queries. Solr's schema provides an idea of how content is structured (more on the schema later), but without documents there is nothing to find. Solr needs input before it can do much.

You may want to add a few sample documents before trying to index your own content. The Solr installation

comes with different types of example documents located under the sub-directories of the example/ directory of your installation.

In the bin/ directory is the post script, a command line tool which can be used to index different types of documents. Do not worry too much about the details for now. The Indexing and Basic Data Operations section has all the details on indexing.

To see some information about the usage of bin/post, use the -help option. Windows users, see the section for Post Tool on Windows.

bin/post can post various types of content to Solr, including files in Solr's native XML and JSON formats, CSV files, a directory tree of rich documents, or even a simple short web crawl. See the examples at the end of

`bin/post -help` for various commands to easily get started posting your content into Solr. Go ahead and add all the documents in some example XML files:

$ bin/post -c gettingstarted example/exampledocs/*.xml SimplePostTool version 5.0.0

Posting files to [base] url http://localhost:8983/solr/gettingstarted/update... Entering auto mode. File endings considered are xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt

,log

POSTing file gb18030-example.xml (application/xml) to [base] POSTing file hd.xml (application/xml) to [base]

POSTing file ipod_other.xml (application/xml) to [base] POSTing file ipod_video.xml (application/xml) to [base] POSTing file manufacturers.xml (application/xml) to [base] POSTing file mem.xml (application/xml) to [base]

POSTing file money.xml (application/xml) to [base] POSTing file monitor.xml (application/xml) to [base] POSTing file monitor2.xml (application/xml) to [base] POSTing file mp500.xml (application/xml) to [base] POSTing file sd500.xml (application/xml) to [base] POSTing file solr.xml (application/xml) to [base]

POSTing file utf8-example.xml (application/xml) to [base] POSTing file vidcard.xml (application/xml) to [base]

14 files indexed.

COMMITting Solr index changes to http://localhost:8983/solr/gettingstarted/update... Time spent: 0:00:00.153

That's it! Solr has indexed the documents contained in those files.

Ask Questions

Now that you have indexed documents, you can perform queries. The simplest way is by building a URL that includes the query parameters. This is exactly the same as building any other HTTP URL.

For example, the following query searches all document fields for "video":

http://localhost:8983/solr/gettingstarted/select?q=video

Notice how the URL includes the host name (localhost), the port number where the server is listening (8983), the application name (solr), the request handler for queries (select), and finally, the query itself (q=video).

The results are contained in an XML document, which you can examine directly by clicking on the link above. The document contains two parts. The first part is the responseHeader, which contains information about the response itself. The main part of the reply is in the result tag, which contains one or more doc tags, each of

which contains fields from documents that match the query. You can use standard XML transformation techniques to mold Solr's results into a form that is suitable for displaying to users. Alternatively, Solr can output the results in JSON, PHP, Ruby and even user-defined formats.

Just in case you are not running Solr as you read, the following screen shot shows the result of a query (the next example, actually) as viewed in Mozilla Firefox. The top-level response contains a lst named responseHeade r and a result named response. Inside result, you can see the three docs that represent the search results.

An XML response to a query.

Once you have mastered the basic idea of a query, it is easy to add enhancements to explore the query syntax. This one is the same as before but the results only contain the ID, name, and price for each returned document. If you don't specify which fields you want, all of them are returned.

http://localhost:8983/solr/gettingstarted/select?q=video&fl=id,name,price

Here is another example which searches for "black" in the name field only. If you do not tell Solr which field to search, it will search default fields, as specified in the schema.

http://localhost:8983/solr/gettingstarted/select?q=name:black

You can provide ranges for fields. The following query finds every document whose price is between $0 and

$400.

http://localhost:8983/solr/gettingstarted/select?q=price:[0%20TO%20400]&fl=id,name

,price

Faceted browsing is one of Solr's key features. It allows users to narrow search results in ways that are meaningful to your application. For example, a shopping site could provide facets to narrow search results by manufacturer or price.

Faceting information is returned as a third part of Solr's query response. To get a taste of this power, take a look at the following query. It adds facet=true and facet.field=cat.

http://localhost:8983/solr/gettingstarted/select?q=price:[0%20TO%20400]&fl=id,name

,price&facet=true&facet.field=cat

In addition to the familiar responseHeader and response from Solr, a facet_counts element is also present. Here is a view with the responseHeader and response collapsed so you can see the faceting information clearly.

An XML Response with faceting

<response>

<lst name="responseHeader">

...

</lst>

<result name="response" numFound="9" start="0">

<doc>

<str name="id">SOLR1000</str>

<str name="name">Solr, the Enterprise Search Server</str>

<float name="price">0.0</float></doc>

...

</result>

<lst name="facet_counts">

<lst name="facet_queries"/>

<lst name="facet_fields">

<lst name="cat">

<int name="electronics">6</int>

<int name="memory">3</int>

<int name="search">2</int>

<int name="software">2</int>

<int name="camera">1</int>

<int name="copier">1</int>

<int name="multifunction printer">1</int>

<int name="music">1</int>

<int name="printer">1</int>

<int name="scanner">1</int>

<int name="connector">0</int>

<int name="currency">0</int>

<int name="graphics card">0</int>

<int name="hard drive">0</int>

<int name="monitor">0</int>

</lst>

</lst>

<lst name="facet_dates"/>

<lst name="facet_ranges"/>

</lst>

</response>

The facet information shows how many of the query results have each possible value of the cat field. You could easily use this information to provide users with a quick way to narrow their query results. You can filter results by adding one or more filter queries to the Solr request. This request constrains documents with a category of "software".

http://localhost:8983/solr/gettingstarted/select?q=price:0%20TO%20400&fl=id,name,p rice&facet=true&facet.field=cat&fq=cat:software

A Quick Overview‌

Having had some fun with Solr, you will now learn about all the cool things it can do. Here is a example of how Solr might be integrated into an application:

In the scenario above, Solr runs along side other server applications. For example, an online store application would provide a user interface, a shopping cart, and a way to make purchases for end users; while an inventory management application would allow store employees to edit product information. The product metadata would be kept in some kind of database, as well as in Solr.

Solr makes it easy to add the capability to search through the online store through the following steps:

1. Define a schema. The schema tells Solr about the contents of documents it will be indexing. In the online store example, the schema would define fields for the product name, description, price, manufacturer, and so on. Solr's schema is powerful and flexible and allows you to tailor Solr's behavior to your application. See Documents, Fields, and Schema Design for all the details.

2. Deploy Solr.

3. Feed Solr documents for which your users will search.

4. Expose search functionality in your application.

Because Solr is based on open standards, it is highly extensible. Solr queries are RESTful, which means, in essence, that a query is a simple HTTP request URL and the response is a structured document: mainly XML, but it could also be JSON, CSV, or some other format. This means that a wide variety of clients will be able to use Solr, from other web applications to browser clients, rich client applications, and mobile devices. Any platform capable of HTTP can talk to Solr. See Client APIs for details on client APIs.

Solr is based on the Apache Lucene project, a high-performance, full-featured search engine. Solr offers support for the simplest keyword searching through to complex queries on multiple fields and faceted search results. Sea rching has more information about searching and queries.

If Solr's capabilities are not impressive enough, its ability to handle very high-volume applications should do the trick.

A relatively common scenario is that you have so much data, or so many queries, that a single Solr server is unable to handle your entire workload. In this case, you can scale up the capabilities of your application using So lrCloud to better distribute the data, and the processing of requests, across many servers. Multiple options can be mixed and matched depending on the type of scalability you need.

For example: "Sharding" is a scaling technique in which a collection is split into multiple logical pieces called "shards" in order to scale up the number of documents in a collection beyond what could physically fit on a single server. Incoming queries are distributed to every shard in the collection, which respond with merged results.

Another technique available is to increase the "Replication Factor" of your collection, which allows you to add

servers with additional copies of your collection to handle higher concurrent query load by spreading the requests around to multiple machines. Sharding and Replication are not mutually exclusive, and together make Solr an extremely powerful and scalable platform.

Best of all, this talk about high-volume applications is not just hypothetical: some of the famous Internet sites that use Solr today are Macy's, EBay, and Zappo's.

For more information, take a look at https://wiki.apache.org/solr/PublicServers.

A Step Closer‌

You already have some idea of Solr's schema. This section describes Solr's home directory and other configuration options.

When Solr runs in an application server, it needs access to a home directory. The home directory contains important configuration information and is the place where Solr will store its index. The layout of the home directory will look a little different when you are running Solr in standalone mode vs when you are running in SolrCloud mode.

The crucial parts of the Solr home directory are shown in these examples:

Standalone Mode

<solr-home-directory>/ solr.xml core_name1/

core.properties conf/

solrconfig.xml managed-schema

data/ core_name2/

core.properties conf/

solrconfig.xml managed-schema

data/

SolrCloud Mode

<solr-home-directory>/ solr.xml core_name1/

core.properties data/

core_name2/ core.properties data/

You may see other files, but the main ones you need to know are:

solr.xml specifies configuration options for your Solr server instance. For more information on solr.xm l see Solr Cores and solr.xml.

Per Solr Core:

core.properties defines specific properties for each core such as its name, the collection the core belongs to, the location of the schema, and other parameters. For more details on core.pro perties, see the section Defining core.properties.

solrconfig.xml controls high-level behavior. You can, for example, specify an alternate location for the data directory. For more information on solrconfig.xml, see Configuring solrconfig.xml. managed-schema (or schema.xml instead) describes the documents you will ask Solr to index. The Schema define a document as a collection of fields. You get to define both the field types and the fields themselves. Field type definitions are powerful and include information about how Solr processes incoming field values and query values. For more information on Solr Schemas, see Doc uments, Fields, and Schema Design and the Schema API.

data/ The directory containing the low level index files.

Note that the SolrCloud example does not include a conf directory for each Solr Core (so there is no solrconf ig.xml or Schema file). This is because the configuration files usually found in the conf directory are stored in ZooKeeper so they can be propagated across the cluster.

If you are using SolrCloud with the embedded ZooKeeper instance, you may also see zoo.cfg and zoo.data which are ZooKeeper configuration and data files. However, if you are running your own ZooKeeper ensemble, you would supply your own ZooKeeper configuration file when you start it and the copies in Solr would be

unused. For more information about ZooKeeper and SolrCloud, see the section SolrCloud.

Solr Start Script Reference‌

Solr includes a script known as "bin/solr" that allows you to start and stop Solr, create and delete collections or cores, and check the status of Solr and configured shards. You can find the script in the bin/ directory of your Solr installation. The bin/solr script makes Solr easier to work with by providing simple commands and options to quickly accomplish common goals.

In this section, the headings below correspond to available commands. For each command, the available options are described with examples.

More examples of bin/solr in use are available throughout the Solr Reference Guide, but particularly in the sections Running Solr and Getting Started with SolrCloud.

Starting and Stopping

Start and Restart Stop

Informational

Version Status Healthcheck

Collections and Cores Create

Delete ZooKeeper Operations

Uploading a Configuration Set Downloading a Configuration Set

Copy between local files and Zookeeper znodes Remove a znode from Zookeeper

Move one Zookeeper znode to another (rename) List a Zookeeper znode's children

Starting and Stopping

Start and Restart

The start command starts Solr. The restart command allows you to restart Solr while it is already running or if it has been stopped already.

The start and restart commands have several options to allow you to run in SolrCloud mode, use an example configuration set, start with a hostname or port that is not the default and point to a local ZooKeeper ensemble.

bin/solr start [options] bin/solr start -help bin/solr restart [options] bin/solr restart -help

When using the restart command, you must pass all of the parameters you initially passed when you started

Solr. Behind the scenes, a stop request is initiated, so Solr will be stopped before being started again. If no nodes are already running, restart will skip the step to stop and proceed to starting Solr.

Available Parameters

The bin/solr script provides many options to allow you to customize the server in common ways, such as changing the listening port. However, most of the defaults are adequate for most Solr installations, especially when just getting started.

To emphasize how the default settings work take a moment to understand that the following commands are equivalent:

bin/solr start

bin/solr start -h localhost -p 8983 -d server -s solr -m 512m

It is not necessary to define all of the options when starting if the defaults are fine for your needs.

Setting Java System Properties

The bin/solr script will pass any additional parameters that begin with -D to the JVM, which allows you to set arbitrary Java system properties. For example, to set the auto soft-commit frequency to 3 seconds, you can do:

bin/solr start -Dsolr.autoSoftCommit.maxTime=3000

SolrCloud Mode

The -c and -cloud options are equivalent:

bin/solr start -c bin/solr start -cloud

If you specify a ZooKeeper connection string, such as -z 192.168.1.4:2181, then Solr will connect to

ZooKeeper and join the cluster. If you do not specify the -z option when starting Solr in cloud mode, then Solr will launch an embedded ZooKeeper server listening on the Solr port + 1000, i.e., if Solr is running on port 8983, then the embedded ZooKeeper will be listening on port 9983.

IMPORTANT: If your ZooKeeper connection string uses a chroot, such as localhost:2181/solr, then you need to bootstrap the /solr znode before launching SolrCloud using the bin/solr script. To do this, you need to use the zkcli.sh script shipped with Solr, such as:

server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181/solr -cmd bootstrap

-solrhome server/solr

When starting in SolrCloud mode, the interactive script session will prompt you to choose a configset to use.

For more information about starting Solr in SolrCloud mode, see also the section Getting Started with SolrCloud.

Running with Example Configurations

bin/solr start -e <name>

The example configurations allow you to get started quickly with a configuration that mirrors what you hope to accomplish with Solr.

Each example launches Solr in with a managed schema, which allows use of the Schema API to make schema edits, but does not allow manual editing of a Schema file If you would prefer to manually modify a schema.xml fi le directly, you can change this default as described in the section Schema Factory Definition in SolrConfig.

Unless otherwise noted in the descriptions below, the examples do not enable SolrCloud nor schemaless mode. The following examples are provided:

cloud: This example starts a 1-4 node SolrCloud cluster on a single machine. When chosen, an interactive session will start to guide you through options to select the initial configset to use, the number of nodes for your example cluster, the ports to use, and name of the collection to be created. When using this example, you can choose from any of the available configsets found in $SOLR_HOME/server/solr

/configsets.

techproducts: This example starts Solr in standalone mode with a schema designed for the sample documents included in the $SOLR_HOME/example/exampledocs directory. The configset used can be found in $SOLR_HOME/server/solr/configsets/sample_techproducts_configs.

dih: This example starts Solr in standalone mode with the DataImportHandler (DIH) enabled and several example dataconfig.xml files pre-configured for different types of data supported with DIH (such as, database contents, email, RSS feeds, etc.). The configset used is customized for DIH, and is found in $SO LR_HOME/example/example-DIH/solr/conf. For more information about DIH, see the section Uploa ding Structured Data Store Data with the Data Import Handler.

schemaless: This example starts Solr in standalone mode using a managed schema, as described in the section Schema Factory Definition in SolrConfig, and provides a very minimal pre-defined schema. Solr will run in Schemaless Mode with this configuration, where Solr will create fields in the schema on the fly and will guess field types used in incoming documents. The configset used can be found in $SOLR_HOME

/server/solr/configsets/data_driven_schema_configs.

The run in-foreground option (-f) does not work with the -e option since the script needs to perform additional tasks after starting the Solr server.

Stop

The stop command sends a STOP request to a running Solr node, which allows it to shutdown gracefully. The command will wait up to 5 seconds for Solr to stop gracefully and then will forcefully kill the process (kill -9).

bin/solr stop [options] bin/solr stop -help

Available Parameters

Informational

Version

The version command simply returns the version of Solr currently installed and immediately exists.

$ bin/solr version X.Y.0

Status

The status command displays basic JSON-formatted information for any Solr nodes found running on the local system. The status command uses the SOLR_PID_DIR environment variable to locate Solr process ID files to find running Solr instances; the SOLR_PID_DIR variable defaults to the bin directory.

bin/solr status

The output will include a status of each node of the cluster, as in this example:

Found 2 Solr nodes:

Solr process 39920 running on port 7574

{

"solr_home":"/Applications/Solr/example/cloud/node2/solr/", "version":"X.Y.0",

"startTime":"2015-02-10T17:19:54.739Z",

"uptime":"1 days, 23 hours, 55 minutes, 48 seconds",

"memory":"77.2 MB (%15.7) of 490.7 MB",

"cloud":{ "ZooKeeper":"localhost:9865", "liveNodes":"2",

"collections":"2"}}

Solr process 39827 running on port 8865

{

"solr_home":"/Applications/Solr/example/cloud/node1/solr/", "version":"X.Y.0",

"startTime":"2015-02-10T17:19:49.057Z",

"uptime":"1 days, 23 hours, 55 minutes, 54 seconds",

"memory":"94.2 MB (%19.2) of 490.7 MB",

"cloud":{ "ZooKeeper":"localhost:9865", "liveNodes":"2",

"collections":"2"}}

Healthcheck

The healthcheck command generates a JSON-formatted health report for a collection when running in SolrCloud mode. The health report provides information about the state of every replica for all shards in a collection, including the number of committed documents and its current state.

bin/solr healthcheck [options] bin/solr healthcheck -help

Available Parameters

Below is an example healthcheck request and response using a non-standard ZooKeeper connect string, with 2 nodes running:

$ bin/solr healthcheck -c gettingstarted -z localhost:9865

{

"collection":"gettingstarted", "status":"healthy", "numDocs":0,

"numShards":2, "shards":[

{

"shard":"shard1",

"status":"healthy", "replicas":[

{

"name":"core_node1", "url":"http://10.0.1.10:8865/solr/gettingstarted_shard1_replica2/", "numDocs":0,

"status":"active",

"uptime":"2 days, 1 hours, 18 minutes, 48 seconds",

"memory":"25.6 MB (%5.2) of 490.7 MB",

"leader":true},

{

"name":"core_node4", "url":"http://10.0.1.10:7574/solr/gettingstarted_shard1_replica1/", "numDocs":0,

"status":"active",

"uptime":"2 days, 1 hours, 18 minutes, 42 seconds",

"memory":"95.3 MB (%19.4) of 490.7 MB"}]},

{

"shard":"shard2",

"status":"healthy", "replicas":[

{

"name":"core_node2", "url":"http://10.0.1.10:8865/solr/gettingstarted_shard2_replica2/", "numDocs":0,

"status":"active",

"uptime":"2 days, 1 hours, 18 minutes, 48 seconds",

"memory":"25.8 MB (%5.3) of 490.7 MB"},

{

"name":"core_node3", "url":"http://10.0.1.10:7574/solr/gettingstarted_shard2_replica1/", "numDocs":0,

"status":"active",

"uptime":"2 days, 1 hours, 18 minutes, 42 seconds",

"memory":"95.4 MB (%19.4) of 490.7 MB",

"leader":true}]}]}

Collections and Cores

The bin/solr script can also help you create new collections (in SolrCloud mode) or cores (in standalone mode), or delete collections.

Create

The create command detects the mode that Solr is running in (standalone or SolrCloud) and then creates a core

or collection depending on the mode.

bin/solr create [options] bin/solr create -help

Available Parameters

Configuration Directories and SolrCloud

Before creating a collection in SolrCloud, the configuration directory used by the collection must be uploaded to ZooKeeper. The create command supports several use cases for how collections and configuration directories work. The main decision you need to make is whether a configuration directory in ZooKeeper should be shared across multiple collections. Let's work through a few examples to illustrate how configuration directories work in SolrCloud.

First, if you don't provide the -d or -n options, then the default configuration ($SOLR_HOME/server/solr/con figsets/data_driven_schema_configs/conf) is uploaded to ZooKeeper using the same name as the collection. For example, the following command will result in the data_driven_schema_configs configuration

being uploaded to /configs/contacts in ZooKeeper:

bin/solr create -c contacts. If you create

another collection, by doing bin/solr create -c contacts2, then another copy of the data_driven_sch ema_configs directory will be uploaded to ZooKeeper under /configs/contacts2. Any changes you make to the configuration for the contacts collection will not affect the contacts2 collection. Put simply, the default behavior creates a unique copy of the configuration directory for each collection you create.

You can override the name given to the configuration directory in ZooKeeper by using the -n option. For instance, the command bin/solr create -c logs -d basic_configs -n basic will upload the serve r/solr/configsets/basic_configs/conf directory to ZooKeeper as /configs/basic.

Notice that we used the -d option to specify a different configuration than the default. Solr provides several built-in configurations under server/solr/configsets. However you can also provide the path to your own configuration directory using the -d option. For instance, the command bin/solr create -c mycoll -d

/tmp/myconfigs, will upload /tmp/myconfigs into ZooKeeper under /configs/mycoll . To reiterate, the configuration directory is named after the collection unless you override it using the -n option.

Other collections can share the same configuration by specifying the name of the shared configuration using the

-n option. For instance, the following command will create a new collection that shares the basic configuration created previously: bin/solr create -c logs2 -n basic.

Data-driven schema and shared configurations

The data_driven_schema_configs schema can mutate as data is indexed. Consequently, we recommend that you do not share data-driven configurations between collections unless you are certain that all collections should inherit the changes made when indexing data into one of the collections.

Delete

The delete command detects the mode that Solr is running in (standalone or SolrCloud) and then deletes the specified core (standalone) or collection (SolrCloud) as appropriate.

bin/solr delete [options] bin/solr delete -help

If running in SolrCloud mode, the delete command checks if the configuration directory used by the collection

you are deleting is being used by other collections. If not, then the configuration directory is also deleted from ZooKeeper. For example, if you created a collection by doing bin/solr create -c contacts, then the delete command bin/solr delete -c contacts will check to see if the /configs/contacts configuratio n directory is being used by any other collections. If not, then the /configs/contacts directory is removed from ZooKeeper.

Available Parameters

ZooKeeper Operations

The bin/solr script allows certain operations affecting Zookeeper. These operations are for SolrCloud mode only.

bin/solr zk [options] bin/solr zk -help

NOTE: Solr should have been started at least once before issuing these commands to initialize Zookeeper with

the znodes Solr expects. Once ZooKeeper is initialized, Solr doesn't need to be running on any node to use these commands.

Uploading a Configuration Set

Use this Zookeeper sub-command to upload one of the pre-configured configuration set or a customized configuration set to Zookeeper.

Available Parameters (all parameters are required)

An example of this command with these parameters is:

bin/solr zk upconfig -z 111.222.333.444:2181 -n mynewconfig -d /path/to/configset

This command does not automatically make changes effective! It simply uploads the configuration sets to Zookeeper. You can use the Collections API to issue a RELOAD command for any collections that uses this configuration set.

Downloading a Configuration Set

Use this Zookeeper sub-command to download a configuration set from Zookeeper to the local filesystem.

Available Parameters (all parameters are required)

An example of this command with the parameters is:

bin/solr zk downconfig -z 111.222.333.444:2181 -n mynewconfig -d /path/to/configset

A "best practice" is to keep your configuration sets in some form of version control as the system-of-record. In that scenario, downconfig should rarely be used.

Copy between local files and Zookeeper znodes

Use this Zookeeper sub-command for transferring files and directories between Zookeeper znodes and your local drive. This command will copy from the local drive to Zookeeper, from Zookeeper to the local drive or from Zookeeper to Zookeeper.

Available Parameters

An example of this command with the parameters is: Recursively copy a directory from local to Zookeeper.

bin/solr zk cp -r file:/apache/confgs/whatever/conf zk:/configs/myconf -z 111.222.333.444:2181

Copy a single file from Zookeeper to local.

bin/solr zk cp zk:/configs/myconf/managed_schema /configs/myconf/managed_schema -z 111.222.333.444:2181

Remove a znode from Zookeeper

Use this ZooKeeper sub-command to remove a znode (and optionally all child nodes) from Zookeeper

Available Parameters

An example of this command with the parameters is:

bin/solr zk rm -r /configs

bin/solr zk rm /configs/myconfigset/schema.xml

Move one Zookeeper znode to another (rename)

Use this ZooKeeper sub-command to move (rename) a Zookeeper znode‌

Available Parameters

An example of this command is:

bin/solr zk mv /configs/oldconfigset /configs/newconfigset

List a Zookeeper znode's children

Use this ZooKeeper sub-command to see the children of a znode.

Available Parameters

An example of this command with the parameters is:

bin/solr zk ls -r /collections/mycollection bin/solr zk ls /collections

Upgrading Solr

If you are already using Solr 6.2, Solr 6.3 should not present any major problems. However, you should review the CHANGES.txt file found in your Solr package for changes and updates that may effect your existing implementation. Detailed steps for upgrading a Solr cluster can be found in the appendix: Upgrading a Solr Cluster.

Upgrading from 6.2.x

If you use the JSON Facet API (json.facet) with method=stream, you must now set sort='index asc' to get the streaming behavior; otherwise it won't stream. Reminder: "method" is a hint that doesn't change defaults of other parameters.

If you use the JSON Facet API (json.facet) to facet on a numeric field and if you use mincount=0 or if you set the prefix, then you will now get an error as these options are incompatible with numeric faceting. Solr's logging verbosity at the INFO level has been greatly reduced, and you may need to update the log configs to use the DEBUG level to see all the logging messages you used to see at INFO level before.

We are no longer backing up solr.log and solr_gc.log files in date-stamped copies forever. If you relied on the solr_log_<date> or solr_gc_log_<date> being in the logs folder that will no longer be the case. See the section Configuring Logging for details on how log rotation works as of Solr 6.3.

The create/deleteCollection methods on MiniSolrCloudCluster have been deprecated. Clients should instead use the CollectionAdminRequest API. In addition, MiniSolrCloudCluster#uploadConfigDi r(File, String) has been deprecated in favour of #uploadConfigSet(Path, String).

The bin/solr.in.sh (bin/solr.in.cmd on Windows) is now completely commented by default. Previously, this wasn't so, which had the effect of masking existing environment variables.

The _version_ field is no longer indexed and is now defined with indexed=false by default, because the field has DocValues enabled.

Upgrading from earlier 6.x versions

If you use historical dates, specifically on or before the year 1582, you should re-index after upgrading to this version.

Upgrading from 5.5.x

The deprecated SolrServer and subclasses have been removed, use SolrClient instead.

The deprecated <nrtMode> configuration in solrconfig.xml has been removed. Please remove it from solrconfig.xml.

SolrClient.shutdown() has been removed, use SolrClient.close() instead.

The deprecated zkCredientialsProvider element in solrcloud section of solr.xml is now removed. Use the correct spelling ( zkCredentialsProvider ) instead.

Internal/expert - ResultContext was significantly changed and expanded to allow for multiple full query results (DocLists) per Solr request. TransformContext was rendered redundant and was removed. See SOLR-7957 for details.

Several changes have been made regarding the " Similarity " used in Solr, in order to provide better default behavior for new users. There are 3 key impacts of these changes on existing users who upgrade:

DefaultSimilarityFactory has been removed. If you currently have DefaultSimilarityF actory explicitly referenced in your schema.xml, edit your config to use the functionally identical ClassicSimilarityFactory. See SOLR-8239 for more details.

The implicit default Similarity used when no <similarity/> is configured in schema.xml has been changed to SchemaSimilarityFactory. Users who wish to preserve back-compatible

behavior should either explicitly configure ClassicSimilarityFactory, or ensure that the luc eneMatchVersion for the collection is less then 6.0. See SOLR-8270 + SOLR-8271 for details. SchemaSimilarityFactory has been modified to use BM25Similarity as the default for fie ldTypes that do not explicitly declare a Similarity. The legacy behavior of using ClassicSimila rity as the default will occur if the luceneMatchVersion for the collection is less then 6.0, or the 'defaultSimFromFieldType' configuration option may be used to specify any default of your choosing. See SOLR-8261 + SOLR-8329 for more details.‌

If your solrconfig.xml file doesn't explicitly mention the schemaFactory to use then Solr will choose the ManagedIndexSchemaFactory by default. Previously it would have chosen ClassicIndexSchem aFactory. This means that the Schema APIs (/<collection>/schema) are enabled and the schema is mutable. When Solr starts your schema.xml file will be renamed to managed-schema. If you want to retain the old behaviour then please ensure that the solrconfig.xml explicitly uses the ClassicInde xSchemaFactory or your luceneMatchVersion in the solrconfig.xml is less than 6.0. See the Sc hema Factory Definition in SolrConfig section for more details

SolrIndexSearcher.QueryCommand and QueryResult were moved to their own classes. If you reference them in your code, you should import them under o.a.s.search (or use your IDE's "Organize Imports").

The ' useParams ' attribute specified in request handler cannot be overridden from request params. See SOLR-8698 for more details.

When requesting stats in date fields, "sum" is now returned as a double value instead of a date. See SOL R-8671 for more details.

The deprecated GET methods for schema are now accessible through the bulk API. These methods now accept fewer request parameters, and output less information. See SOLR-8736 for more details. Some of the removed functionality will likely be restored in a future version of Solr - see SOLR-8992.

In the past, Solr guaranteed that retrieval of multi-valued fields would preserve the order of values. Because values may now be retrieved from column-stored fields (docValues="true"), in conjunction with the fact that DocValues do not currently preserve order, means that users should set useDocValues AsStored="false" to prevent future optimizations from using the column-stored values over the

row-stored values when fields have both stored="true" and docValues="true".

Formatted date-times from Solr have some differences. If the year is more than 4 digits, there is a leading '+'. When there is a non-zero number of milliseconds, it is padded with zeros to 3 digits. Negative year (BC) dates are now possible. Parsing: It is now an error to supply a portion of the date out of its, range, like 67 seconds.

SolrJ no longer includes DateUtil. If for some reason you need to format or parse dates, simply use In stant.format() and Instant.parse().

If you are using spatial4j, please upgrade to 0.6 and edit your spatialContextFactory to replace com

.spatial4j.core with org.locationtech.spatial4j .

Upgrading from Older Versions of Solr

Users upgrading from older versions are strongly encouraged to consult CHANGES.txt for the details of all cha nges since the version they are upgrading from.

A summary of the significant changes between Solr 5.x and Solr 6.0 can be found in the Major Changes from Solr 5 to Solr 6 section.

Using the Solr Administration User Interface

This section discusses the Solr Administration User Interface ("Admin UI").

The Overview of the Solr Admin UI explains the basic features of the user interface, what's on the initial Admin UI page, and how to configure the interface. In addition, there are pages describing each screen of the Admin UI:

Getting Assistance shows you how to get more information about the UI.

Logging >shows recent messages logged by this Solr node and provides a way to change logging levels for specific classes.

Cloud Screens display information about nodes when running in SolrCloud mode. Collections / Core Admin explains how to get management information about each core. Java Properties shows the Java information about each core.

Thread Dump lets you see detailed information about each thread, along with state information.

Collection-Specific Tools is a section explaining additional screens available for each collection.

Analysis - lets you analyze the data found in specific fields.

Dataimport - shows you information about the current status of the Data Import Handler. Documents - provides a simple form allowing you to execute various Solr indexing commands directly from the browser.

Files - shows the current core configuration files such as solrconfig.xml. Query - lets you submit a structured query about various elements of a core.

Stream - allows you to submit streaming expressions and see results and parsing explanations. Schema Browser - displays schema data in a browser window.

Core-Specific Tools is a section explaining additional screens available for each named core.

Ping - lets you ping a named core and determine whether the core is active. Plugins/Stats - shows statistics for plugins and other installed components.

Replication - shows you the current replication status for the core, and lets you enable/disable replication.

Segments Info - Provides a visualization of the underlying Lucene index segments.

Overview of the Solr Admin UI‌

Solr features a Web interface that makes it easy for Solr administrators and programmers to view Solr configuration details, run queries and analyze document fields in order to fine-tune a Solr configuration and access online documentation and other help.

Accessing the URL http://hostname:8983/solr/ will show the main dashboard, which is divided into two parts.

A left-side of the screen is a menu under the Solr logo that provides the navigation through the screens of the UI. The first set of links are for system-level information and configuration and provide access to Logging, Collection/ Core Administration and Java Properties, among other things. At the end of this information is at least one pulldown listing Solr cores configured for this instance. On SolrCloud nodes, an additional pulldown list shows all collections in this cluster. Clicking on a collection or core name shows secondary menus of information for the specified collection or core, such as a Schema Browser, Config Files, Plugins & Statistics, and an ability to perform Queries on indexed data.

The center of the screen shows the detail of the option selected. This may include a sub-navigation for the option or text or graphical representation of the requested data. See the sections in this guide for each screen for more details.

Under the covers, the Solr Admin UI re-uses the same HTTP APIs available to all clients to access Solr-related data to drive an external interface.

The path to the Solr Admin UI given above is http://hostname:port/solr, which redirects to http

://hostname:port/solr/#/ in the current version. A convenience redirect is also supported, so simply accessing the Admin UI at http://hostname:port/ will also redirect to http://hostname: port/solr/#/.

Related Topics

Configuring solrconfig.xml

Getting Assistance‌

At the bottom of each screen of the Admin UI is a set of links that can be used to get more assistance with configuring and using Solr.

Assistance icons

These icons include the following links.

Solr Query Syntax

Navigates to the section "Query Syntax and Parsing" in this reference guide.

These links cannot be modified without editing the admin.html in the solr.war that contains the Admin UI files.

Logging‌

The Logging page shows recent messages logged by this Solr node.

When you click the link for "Logging", a page similar to the one below will be displayed:

The Main Logging Screen, including an example of an error due to a bad document sent by a client

While this example shows logged messages for only one core, if you have multiple cores in a single instance, they will each be listed, with the level for each.

Selecting a Logging Level

When you select the Level link on the left, you see the hierarchy of classpaths and classnames for your instance. A row highlighted in yellow indicates that the class has logging capabilities. Click on a highlighted row, and a menu will appear to allow you to change the log level for that class. Characters in boldface indicate that the class will not be affected by level changes to root.

For an explanation of the various logging levels, see Configuring Logging.

Cloud Screens‌

When running in SolrCloud mode, a "Cloud" option will appear in the Admin UI between Logging and Collections/ Core Admin which provides status information about each collection & node in your cluster, as well as access to the low level data being stored in Zookeeper.

Only Visible When using SolrCloud

The "Cloud" menu option is only available on Solr instances running in SolrCloud mode. Single node or master/slave replication instances of Solr will not display this option.

Click on the Cloud option in the left-hand navigation, and a small sub-menu appears with options called "Tree", "Graph", "Graph (Radial)" and "Dump". The default view ("Graph") shows a graph of each collection, the shards that make up those collections, and the addresses of each replica for each shard.

This example shows the very simple two-node cluster created using the "bin/solr -e cloud -noprompt" example command. In addition to the 2 shard, 2 replica "gettingstarted" collection, there is an additional "films" collection consisting of a single shard/replica:

The "Graph (Radial)" option provides a different visual view of each node. Using the same example cluster, the radial graph view looks like:

The "Tree" option shows a directory structure of the data in ZooKeeper, including cluster wide information regarding the live_nodes and overseer status, as well as collection specific information such as the state. json, current shard leaders, and configuration files in use. In this example, we see the state.json file definition for the "films" collection:

The final option is "Dump", which returns a JSON document containing all nodes, their contents and their children (recursively). This can be used to export a snapshot of all the data that Solr has kept inside ZooKeeper and can aid in debugging SolrCloud problems.

Collections / Core Admin‌

The Collections screen provides some basic functionality for managing your Collections, powered by the Collecti ons API.

If you are running a single node Solr instance, you will not see a Collections option in the left nav menu of the Admin UI.

You will instead see a "Core Admin" screen that supports some comparable Core level information & manipulation via the CoreAdmin API instead.

The main display of this page provides a list of collections that exist in your cluster. Clicking on a collection name provides some basic metadata about how the collection is defined, and its current shards & replicas, with options for adding and deleting individual replicas.

The buttons at the top of the screen let you make various collection level changes to your cluster, from add new collections or aliases to reloading or deleting a single collection.

Java Properties‌

The Java Properties screen provides easy access to one of the most essential components of a top-performing Solr systems. With the Java Properties screen, you can see all the properties of the JVM running Solr, including the class paths, file encodings, JVM memory settings, operating system, and more.

Thread Dump‌

The Thread Dump screen lets you inspect the currently active threads on your server. Each thread is listed and access to the stacktraces is available where applicable. Icons to the left indicate the state of the thread: for example, threads with a green check-mark in a green circle are in a "RUNNABLE" state. On the right of the thread name, a down-arrow means you can expand to see the stacktrace for that thread.

When you move your cursor over a thread name, a box floats over the name with the state for that thread. Thread states can be:

State Meaning

When you click on one of the threads that can be expanded, you'll see the stacktrace, as in the example below:

Inspecting a thread

You can also check the Show all Stacktraces button to automatically enable expansion for all threads.

Collection-Specific Tools‌

In the left-hand navigation bar, you will see a pull-down menu titled "Collection Selector" that can be used to access collection specific administration screens.

Only Visible When using SolrCloud

The "Collection Selector" pull-down menu is only available on Solr instances running in SolrCloud mode.

Single node or master/slave replication instances of Solr will not display this menu, instead the Collection specific UI pages described in this section will be available in the Core Selector pull-down menu.

Clicking on the Collection Selector pull-down menu will show a list of the collections in your Solr cluster, with a search box that can be used to find a specific collection by name. When you select a collection from the

pull-down, the main display of the page will display some basic metadata about the collection, and a secondary menu will appear in the left nav with links to additional collection specific administration screens.

The collection-specific UI screens are listed below, with a link to the section of this guide to find out more:

Analysis - lets you analyze the data found in specific fields.

Dataimport - shows you information about the current status of the Data Import Handler.

Documents - provides a simple form allowing you to execute various Solr indexing commands directly from the browser.

Files - shows the current core configuration files such as solrconfig.xml. Query - lets you submit a structured query about various elements of a core.

Stream - allows you to submit streaming expressions and see results and parsing explanations. Schema Browser - displays schema data in a browser window.

Analysis Screen‌

The Analysis screen lets you inspect how data will be handled according to the field, field type and dynamic field configurations found in your Schema. You can analyze how content would be handled during indexing or during query processing and view the results separately or at the same time. Ideally, you would want content to be handled consistently, and this screen allows you to validate the settings in the field type or field analysis chains.

Enter content in one or both boxes at the top of the screen, and then choose the field or field type definitions to use for analysis.

If you click the Verbose Output check box, you see more information, including more details on the transformations to the input (such as, convert to lower case, strip extra characters, etc.) including the raw bytes, type and detailed position information at each stage. The information displayed will vary depending on the settings of the field or field type. Each step of the process is displayed in a separate section, with an abbreviation for the tokenizer or filter that is applied in that step. Hover or click on the abbreviation, and you'll see the name and path of the tokenizer or filter.

In the example screenshot above, several transformations are applied to the input "Running is a sport." The words "is" and "a" have been removed and the word "running" has been changed to its basic form, "run". This is because we are using the field type text_en in this scenario, which is configured to remove stop words (small words that usually do not provide a great deal of context) and "stem" terms when possible to find more possible matches (this is particularly helpful with plural forms of words). If you click the question mark next to the Analyze Fieldname/Field Type pull-down menu, the Schema Browser window will open, showing you the settings for the field specified.

The section Understanding Analyzers, Tokenizers, and Filters describes in detail what each option is and how it may transform your data and the section Running Your Analyzer has specific examples for using the Analysis screen.

Dataimport Screen‌

The Dataimport screen shows the configuration of the DataImportHandler (DIH) and allows you start, and monitor the status of, import commands as defined by the options selected on the screen and defined in the configuration file.

This screen also lets you adjust various options to control how the data is imported to Solr, and view the data import configuration file that controls the import. For more information about data importing with DIH, see the section on Uploading Structured Data Store Data with the Data Import Handler.

Documents Screen‌

The Documents screen provides a simple form allowing you to execute various Solr indexing commands in a variety of formats directly from the browser.

The screen allows you to:

Copy documents in JSON, CSV or XML and submit them to the index Upload documents (in JSON, CSV or XML)

Construct documents by selecting fields and field values

The first step is to define the RequestHandler to use (aka, 'qt'). By default /update will be defined. To use Solr Cell, for example, change the request handler to /update/extract.

Then choose the Document Type to define the type of document to load. The remaining parameters will change depending on the document type selected.

JSON

When using the JSON document type, the functionality is similar to using a requestHandler on the command line. Instead of putting the documents in a curl command, they can instead be input into the Document entry box. The document structure should still be in proper JSON format.

Then you can choose when documents should be added to the index (Commit Within), whether existing documents should be overwritten with incoming documents with the same id (if this is not true, then the incoming documents will be dropped), and, finally, if a document boost should be applied.

This option will only add or overwrite documents to the index; for other update tasks, see the Solr Command opti on.‌

CSV

When using the CSV document type, the functionality is similar to using a requestHandler on the command line. Instead of putting the documents in a curl command, they can instead be input into the Document entry box. The document structure should still be in proper CSV format, with columns delimited and one row per document.

Then you can choose when documents should be added to the index (Commit Within), and whether existing documents should be overwritten with incoming documents with the same id (if this is not true, then the incoming documents will be dropped).

Document Builder

The Document Builder provides a wizard-like interface to enter fields of a document

File Upload

The File Upload option allows choosing a prepared file and uploading it. If using only /update for the Request-Handler option, you will be limited to XML, CSV, and JSON.

However, to use the ExtractingRequestHandler (aka Solr Cell), you can modify the Request-Handler to /update

/extract. You must have this defined in your solrconfig.xml file, with your desired defaults. You should also update the &literal.id shown in the Extracting Req. Handler Params so the file chosen is given a unique id.

Then you can choose when documents should be added to the index (Commit Within), and whether existing documents should be overwritten with incoming documents with the same id (if this is not true, then the incoming documents will be dropped).

Solr Command

The Solr Command option allows you use XML or JSON to perform specific actions on documents, such as defining documents to be added or deleted, updating only certain fields of documents, or commit and optimize commands on the index.

The documents should be structured as they would be if using /update on the command line.

XML

When using the XML document type, the functionality is similar to using a requestHandler on the command line. Instead of putting the documents in a curl command, they can instead be input into the Document entry box. The document structure should still be in proper Solr XML format, with each document separated by <doc> tags and each field defined.

Then you can choose when documents should be added to the index (Commit Within), and whether existing documents should be overwritten with incoming documents with the same id (if this is not true, then the incoming documents will be dropped).

This option will only add or overwrite documents to the index; for other update tasks, see the Solr Command opti on.

Related Topics

Uploading Data with Index Handlers

Uploading Data with Solr Cell using Apache Tika

Files Screen

The Files screen lets you browse & view the various configuration files (such solrconfig.xml and the schema file) for the collection you selected.

If you are using SolrCloud, the files displayed are the configuration files for this collection stored in ZooKeeper (using upconfig), for single node Solr installations, all files in the ./conf directory are displayed.

While solrconfig.xml defines the behaviour of Solr as it indexes content and responds to queries, the Schema allows you to define the types of data in your content (field types), the fields your documents will be broken into, and any dynamic fields that should be generated based on patterns of field names in the incoming documents. Any other configuration files are used depending on how they are referenced in either solrconfig

.xml or your schema.

Configuration files cannot be edited with this screen, so a text editor of some kind must be used.

This screen is related to the Schema Browser Screen, in that they both can display information from the schema, but the Schema Browser provides a way to drill into the analysis chain and displays linkages between field types, fields, and dynamic field rules.

Many of the options defined in these configuration files are described throughout the rest of this Guide. In particular, you will want to review these sections:

Indexing and Basic Data Operations Searching

The Well-Configured Solr Instance Documents, Fields, and Schema Design

Query Screen‌

You can use the Query screen to submit a search query to a Solr collection and analyze the results. In the

example in the screenshot, a query has been submitted, and the screen shows the query results sent to the browser as JSON.

In this example a query for genre:Fantasy was sent to a "films" collection. Defaults were used for all other options in the form, which are explained briefly in the table below, and covered in detail in later parts of this Guide.

The response is shown to the right of the form. Requests to Solr are simply HTTP requests, and the query submitted is shown in light type above the results; if you click on this it will open a new browser window with just this request and response (without the rest of the Solr Admin UI). The rest of the response is shown in JSON, which is part of the request (see the wt=json part at the end).

The response has at least two sections, but may have several more depending on the options chosen. The two sections it always has are the responseHeader and the response. The responseHeader includes the status of the search (status), the processing time (QTime), and the parameters (params) that were used to process the query.

The response includes the documents that matched the query, in doc sub-sections. The fields return depend on the parameters of the query (and the defaults of the request handler used). The number of results is also included in this section.

This screen allows you to experiment with different query options, and inspect how your documents were indexed. The query parameters available on the form are some basic options that most users want to have available, but there are dozens more available which could be simply added to the basic request by hand (if opened in a browser). The table below explains the parameters available:

Related Topics

Searching

Stream Screen‌

The Stream screen allows you to enter a streaming expression and see the results. It is very similar to the Query Screen, except the input box is at the top and all options must be declared in the expression.

The screen will insert everything up to the streaming expression itself, so you do not need to enter the full URI with the hostname, port, collection, etc. Simply input the expression after the `expr=` part, and the URL will be constructed dynamically as appropriate.

Under the input box, the Execute button will run the expression. An option "with explanation" will show the parts of the streaming expression that were executed. Under this, the streamed results are shown. A URL to be able to

view the output in a browser is also available.

Schema Browser Screen‌

The Schema Browser screen lets you review schema data in a browser window. If you have accessed this window from the Analysis screen, it will be opened to a specific field, dynamic field rule or field type. If there is nothing chosen, use the pull-down menu to choose the field or field type.

The screen provides a great deal of useful information about each particular field and fieldtype in the Schema, and provides a quick UI for adding fields or fieldtypes using the Schema API (if enabled). In the example above, we have chosen the cat field. On the left side of the main view window, we see the field name, that it is copied to the _text_ (because of a copyField rule) and that it use the strings fieldtype. Click on one of those field or fieldtype names, and you can see the corresponding definitions.

In the right part of the main view, we see the specific properties of how the cat field is defined – either explicitly or implicitly via its fieldtype, as well as how many documents have populated this field. Then we see the analyzer used for indexing and query processing. Click the icon to the left of either of those, and you'll see the definitions for the tokenizers and/or filters that are used. The output of these processes is the information you see when testing how content is handled for a particular field with the Analysis Screen.

Under the analyzer information is a button to Load Term Info. Clicking that button will show the top N terms that are in a sample shard for that field, as well as a histogram showing the number of terms with various frequencies. Click on a term, and you will be taken to the Query Screen to see the results of a query of that term in that field. If you want to always see the term information for a field, choose Autoload and it will always appear when there are terms for a field. A histogram shows the number of terms with a given frequency in the field.

Term Information is loaded from single arbitrarily selected core from the collection, to provide a representative sample for the collection. Full Field Facet query results are needed to see precise term counts across the entire collection.

Core-Specific Tools‌

In the left-hand navigation bar, you will see a pull-down menu titled "Core Selector". Clicking on the menu will show a list of Solr cores hosted on this Solr node, with a search box that can be used to find a specific core by name. When you select a core from the pull-down, the main display of the page will display some basic metadata about the core, and a secondary menu will appear in the left nav with links to additional core specific administration screens. You can also define a configuration file called admin-extra.html that includes links or other information you would like to display in the "Admin Extra" part of this main screen.

The core-specific UI screens are listed below, with a link to the section of this guide to find out more: Ping - lets you ping a named core and determine whether the core is active.

Plugins/Stats - shows statistics for plugins and other installed components.

Replication - shows you the current replication status for the core, and lets you enable/disable replication. Segments Info - Provides a visualization of the underlying Lucene index segments.

If you are running a single node instance of Solr, additional UI screens normally displayed on a per-collection bases will also be listed:

Analysis - lets you analyze the data found in specific fields.

Dataimport - shows you information about the current status of the Data Import Handler.

Documents - provides a simple form allowing you to execute various Solr indexing commands directly from the browser.

Files - shows the current core configuration files such as solrconfig.xml. Query - lets you submit a structured query about various elements of a core.

Stream - allows you to submit streaming expressions and see results and parsing explanations. Schema Browser - displays schema data in a browser window.

Ping‌

Choosing Ping under a core name issues a ping request to check whether the core is up and responding to requests.

The search executed by a Ping is configured with the Request Parameters API. See Implicit RequestHandlers for the paramset to use for the /admin/ping endpoint.

The Ping option doesn't open a page, but the status of the request can be seen on the core overview page shown when clicking on a collection name. The length of time the request has taken is displayed next to the Ping option, in milliseconds.

API Examples

While the UI screen makes it easy to see the ping response time, the underlying ping command can be more useful when executed by remote monitoring tools:

Input

http://localhost:8983/solr/<core-name>/admin/ping

This command will ping the core name for a response.

Input

http://localhost:8983/solr/<collection-name>admin/ping?wt=json&distrib=true&indent=t rue

This command will ping all replicas of the given collection name for a response

Sample Output

<response>

<lst name="responseHeader">

<int name="status">0</int>

<int name="QTime">13</int>

<lst name="params">

<str name="q">{!lucene}*:*</str>

<str name="distrib">false</str>

<str name="df">_text_</str>

<str name="rows">10</str>

<str name="echoParams">all</str>

</lst>

</lst>

<str name="status">OK</str>

</response>

Both API calls have the same output. A status=OK indicates that the nodes are responding.

SolrJ Example

SolrPing ping = new SolrPing();

ping.getParams().add("distrib", "true"); //To make it a distributed request against a collection

rsp = ping.process(solrClient, collectionName); int status = rsp.getStatus();

Plugins & Stats Screen‌

The Plugins screen shows information and statistics about the status and performance of various plugins running in each Solr core. You can find information about the performance of the Solr caches, the state of Solr's searchers, and the configuration of Request Handlers and Search Components.

Choose an area of interest on the right, and then drill down into more specifics by clicking on one of the names that appear in the central part of the window. In this example, we've chosen to look at the Searcher stats, from the Core area:

Searcher Statistics

The display is a snapshot taken when the page is loaded. You can get updated status by choosing to either Wat ch Changes or Refresh Values. Watching the changes will highlight those areas that have changed, while

refreshing the values will reload the page with updated information.

Replication Screen‌

The Replication screen shows you the current replication state for the core you have specified. SolrCloud has supplanted much of this functionality, but if you are still using Master-Slave index replication, you can use this screen to:

1. View the replicatable index state. (on a master node)

2. View the current replication status (on a slave node)

   

3. Disable replication. (on a master node)

Caution When Using SolrCloud

When using SolrCloud, do not attempt to disable replication via this screen.

More details on how to configure replication is available in the section called Index Replication.

Segments Info‌

The Segments Info screen lets you see a visualization of the various segments in the underlying Lucene index for this core, with information about the size of each segment – both bytes and in number of documents – as well as other basic metadata about those segments. Most visible is the the number of deleted documents, but you can hover your mouse over the segments to see additional numeric details.

‌

This information may be useful for people to help make decisions about the optimal merge settings for their data.

Documents, Fields, and Schema Design

This section discusses how Solr organizes its data into documents and fields, as well as how to work with a schema in Solr.

This section includes the following topics:

Overview of Documents, Fields, and Schema Design: An introduction to the concepts covered in this section.

Solr Field Types: Detailed information about field types in Solr, including the field types in the default Solr schema.

Defining Fields: Describes how to define fields in Solr.

Copying Fields: Describes how to populate fields with data copied from another field.

Dynamic Fields: Information about using dynamic fields in order to catch and index fields that do not exactly conform to other field definitions in your schema.

Schema API: Use curl commands to read various parts of a schema or create new fields and copyField rules. Other Schema Elements: Describes other important elements in the Solr schema.

Putting the Pieces Together: A higher-level view of the Solr schema and how its elements work together. DocValues: Describes how to create a docValues index for faster lookups.

Schemaless Mode: Automatically add previously unknown schema fields using value-based field type guessing.

Overview of Documents, Fields, and Schema Design‌

The fundamental premise of Solr is simple. You give it a lot of information, then later you can ask it questions and find the piece of information you want. The part where you feed in all the information is called indexing or up dating. When you ask a question, it's called a query.

One way to understand how Solr works is to think of a loose-leaf book of recipes. Every time you add a recipe to the book, you update the index at the back. You list each ingredient and the page number of the recipe you just added. Suppose you add one hundred recipes. Using the index, you can very quickly find all the recipes that use garbanzo beans, or artichokes, or coffee, as an ingredient. Using the index is much faster than looking through each recipe one by one. Imagine a book of one thousand recipes, or one million.

Solr allows you to build an index with many different fields, or types of entries. The example above shows how to build an index with just one field, ingredients. You could have other fields in the index for the recipe's cooking style, like Asian, Cajun, or vegan, and you could have an index field for preparation times. Solr can answer questions like "What Cajun-style recipes that have blood oranges as an ingredient can be prepared in fewer than 30 minutes?"

The schema is the place where you tell Solr how it should build indexes from input documents.

How Solr Sees the World

Solr's basic unit of information is a document, which is a set of data that describes something. A recipe document would contain the ingredients, the instructions, the preparation time, the cooking time, the tools needed, and so on. A document about a person, for example, might contain the person's name, biography, favorite color, and shoe size. A document about a book could contain the title, author, year of publication, number of pages, and so on.

In the Solr universe, documents are composed of fields, which are more specific pieces of information. Shoe size

could be a field. First name and last name could be fields.

Fields can contain different kinds of data. A name field, for example, is text (character data). A shoe size field might be a floating point number so that it could contain values like 6 and 9.5. Obviously, the definition of fields is flexible (you could define a shoe size field as a text field rather than a floating point number, for example), but if you define your fields correctly, Solr will be able to interpret them correctly and your users will get better results when they perform a query.

You can tell Solr about the kind of data a field contains by specifying its field type. The field type tells Solr how to interpret the field and how it can be queried.

When you add a document, Solr takes the information in the document's fields and adds that information to an index. When you perform a query, Solr can quickly consult the index and return the matching documents.

Field Analysis

Field analysis tells Solr what to do with incoming data when building an index. A more accurate name for this process would be processing or even digestion, but the official name is analysis.

Consider, for example, a biography field in a person document. Every word of the biography must be indexed so that you can quickly find people whose lives have had anything to do with ketchup, or dragonflies, or cryptography.

However, a biography will likely contains lots of words you don't care about and don't want clogging up your index—words like "the", "a", "to", and so forth. Furthermore, suppose the biography contains the word "Ketchup", capitalized at the beginning of a sentence. If a user makes a query for "ketchup", you want Solr to tell you about the person even though the biography contains the capitalized word.

The solution to both these problems is field analysis. For the biography field, you can tell Solr how to break apart the biography into words. You can tell Solr that you want to make all the words lower case, and you can tell Solr to remove accents marks.

Field analysis is an important part of a field type. Understanding Analyzers, Tokenizers, and Filters is a detailed description of field analysis.

Solr's Schema File

Solr stores details about the field types and fields it is expected to understand in a schema file. The name and location of this file may vary depending on how you initially configured Solr or if you modified it later.

managed-schema is the name for the schema file Solr uses by default to support making Schema changes at runtime via the Schema API, or Schemaless Mode features. You may explicitly configure the managed schema features to use an alternative filename if you choose, but the contents of the files are still updated automatically by Solr.

schema.xml is the traditional name for a schema file which can be edited manually by users who use the ClassicIndexSchemaFactory.

If you are using SolrCloud you may not be able to find any file by these names on the local filesystem. You will only be able to see the schema through the Schema API (if enabled) or through the Solr Admin UI's Cloud Screens.

Whichever name of the file is being used in your installation, the structure of the file is not changed. However, the way you interact with the file will change. If you are using the managed schema, it is expected that you only interact with the file with the Schema API, and never make manual edits. If you do not use the managed schema, you will only be able to make manual edits to the file, the Schema API will not support any modifications.

Note that if you are not using the Schema API yet you do use SolrCloud, you will need to interact with schema.x ml through ZooKeeper using upconfig and downconfig commands to make a local copy and upload your changes. The options for doing this are described in Solr Start Script Reference and Using ZooKeeper to Manage Configuration Files.

Solr Field Types‌

The field type defines how Solr should interpret data in a field and how the field can be queried. There are many field types included with Solr by default, and they can also be defined locally.

Topics covered in this section:

Field Type Definitions and Properties Field Types Included with Solr

Working with Currencies and Exchange Rates Working with Dates

Working with Enum Fields

Working with External Files and Processes Field Properties by Use Case

Related Topics

SchemaXML-DataTypes FieldType Javadoc

Field Type Definitions and Properties‌

A field type definition can include four types of information: The name of the field type (mandatory)

An implementation class name (mandatory)

If the field type is TextField, a description of the field analysis for the field type

Field type properties - depending on the implementation class, some properties may be mandatory.

Field Type Definitions in schema.xml

Field types are defined in schema.xml. Each field type is defined between fieldType elements. They can optionally be grouped within a types element. Here is an example of a field type definition for a type called tex t_general:

<fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">

<analyzer type="index">

<tokenizer class="solr.StandardTokenizerFactory"/>

<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"

/>

<!-- in this example, we will only use synonyms at query time

<filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>

-->

<filter class="solr.LowerCaseFilterFactory"/>

</analyzer>

<analyzer type="query">

<tokenizer class="solr.StandardTokenizerFactory"/>

<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"

/>

<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>

<filter class="solr.LowerCaseFilterFactory"/>

</analyzer>

</fieldType>

The first line in the example above contains the field type name, text_general, and the name of the implementing class, solr.TextField. The rest of the definition is about field analysis, described in Understand ing Analyzers, Tokenizers, and Filters.

The implementing class is responsible for making sure the field is handled correctly. In the class names in schem a.xml, the string solr is shorthand for org.apache.solr.schema or org.apache.solr.analysis.

Therefore, solr.TextField is really org.apache.solr.schema.TextField..

Field Type Properties

The field type class determines most of the behavior of a field type, but optional properties can also be defined. For example, the following definition of a date field type defines two properties, sortMissingLast and omitNo rms.

<fieldType name="date" class="solr.TrieDateField" sortMissingLast="true" omitNorms="true"/>

The properties that can be specified for a given field type fall into three major categories:

Properties specific to the field type's class. General Properties Solr supports for any field type.

Field Default Properties that can be specified on the field type that will be inherited by fields that use this type instead of the default behavior.

General Properties

Lucene index back-compatibility is only supported for the default codec. If you choose to customize the p ostingsFormat or docValuesFormat in your schema.xml, upgrading to a future version of Solr may require you to either switch back to the default codec and optimize your index to rewrite it into the default codec before upgrading, or re-build your entire index from scratch after upgrading.

Field Default Properties

These are properties that can be specified either on the field types, or on individual fields to override the values provided by the field types. The default values for each property depend on the underlying FieldType class, which in turn may depend on the version attribute of the <schema/>. The table below includes the default value for most FieldType implementations provided by Solr, assuming a schema.xml that declares version

="1.6".

Field Type Similarity

A field type may optionally specify a <similarity/> that will be used when scoring documents that refer to fields with this type, as long as the "global" similarity for the collection allows it. By default, any field type which does not define a similarity, uses BM25Similarity. For more details, and examples of configuring both global

& per-type Similarities, please see Other Schema Elements.

Field Types Included with Solr‌

The following table lists the field types that are available in Solr. The org.apache.solr.schema package includes all the classes listed in this table.

Working with Currencies and Exchange Rates‌

The currency FieldType provides support for monetary values to Solr/Lucene with query-time currency conversion and exchange rates. The following features are supported:

Point queries Range queries

Function range queries Sorting

Currency parsing by either currency code or symbol

Symmetric & asymmetric exchange rates (asymmetric exchange rates are useful if there are fees associated with exchanging the currency)

Configuring Currencies

The currency field type is defined in schema.xml. This is the default configuration of this type:

<fieldType name="currency" class="solr.CurrencyField" precisionStep="8" defaultCurrency="USD" currencyConfig="currency.xml" />

In this example, we have defined the name and class of the field type, and defined the defaultCurrency as "USD", for U.S. Dollars. We have also defined a currencyConfig to use a file called "currency.xml". This is a file of exchange rates between our default currency to other currencies. There is an alternate implementation that would allow regular downloading of currency data. See Exchange Rates below for more.

At indexing time, money fields can be indexed in a native currency. For example, if a product on an e-commerce site is listed in Euros, indexing the price field as "1000,EUR" will index it appropriately. The price should be separated from the currency by a comma, and the price must be encoded with a floating point value (a decimal point).‌

During query processing, range and point queries are both supported.

Exchange Rates

You configure exchange rates by specifying a provider. Natively, two provider types are supported: FileExchan geRateProvider or OpenExchangeRatesOrgProvider.

FileExchangeRateProvider

This provider requires you to provide a file of exchange rates. It is the default, meaning that to use this provider you only need to specify the file path and name as a value for currencyConfig in the definition for this type.

There is a sample currency.xml file included with Solr, found in the same directory as the schema.xml file. Here is a small snippet from this file:

<currencyConfig version="1.0">

<rates>

<!-- Updated from http://www.exchangerate.com/ at 2011-09-27 -->

<rate from="USD" to="ARS" rate="4.333871" comment="ARGENTINA Peso" />

<rate from="USD" to="AUD" rate="1.025768" comment="AUSTRALIA Dollar" />

<rate from="USD" to="EUR" rate="0.743676" comment="European Euro" />

<rate from="USD" to="CAD" rate="1.030815" comment="CANADA Dollar" />

<!-- Cross-rates for some common currencies -->

<rate from="EUR" to="GBP" rate="0.869914" />

<rate from="EUR" to="NOK" rate="7.800095" />

<rate from="GBP" to="NOK" rate="8.966508" />

<!-- Asymmetrical rates -->

<rate from="EUR" to="USD" rate="0.5" />

</rates>

</currencyConfig>

OpenExchangeRatesOrgProvider

You can configure Solr to download exchange rates from OpenExchangeRates.Org, with updates rates between USD and 170 currencies hourly. These rates are symmetrical only.

In this case, you need to specify the providerClass in the definitions for the field type and sign up for an API key. Here is an example:

<fieldType name="currency" class="solr.CurrencyField" precisionStep="8" providerClass="solr.OpenExchangeRatesOrgProvider" refreshInterval="60"

ratesFileLocation="http://www.openexchangerates.org/api/latest.json?app_id=yourPerso nalAppIdKey"/>

The refreshInterval is minutes, so the above example will download the newest rates every 60 minutes. The refresh interval may be increased, but not decreased.

Working with Dates

Date Formatting

Solr's date fields (TrieDateField and DateRangeField) represents a point in time with millisecond precision. The format used is a restricted form of the canonical representation of dateTime in the XML Schema specification – a restricted subset of ISO-8601. For those familiar with Java 8, Solr uses DateTimeFormatter.IS O_INSTANT for formatting, and parsing too with "leniency".

YYYY-MM-DDThh:mm:ssZ YYYY is the year.

MM is the month.

DD is the day of the month.

hh is the hour of the day as on a 24-hour clock.

mm is minutes.

ss is seconds.

Z is a literal 'Z' character indicating that this string representation of the date is in UTC

Note that no time zone can be specified; the String representations of dates is always expressed in Coordinated Universal Time (UTC). Here is an example value:

1972-05-20T17:33:18Z

You can optionally include fractional seconds if you wish, although any precision beyond milliseconds will be ignored. Here are example values with sub-seconds:

1972-05-20T17:33:18.772Z

1972-05-20T17:33:18.77Z

1972-05-20T17:33:18.7Z

There must be a leading '-' for dates prior to year 0000, and Solr will format dates with a leading '+' for years after 9999. Year 0000 is considered year 1 BC; there is no such thing as year 0 AD or BC.

Query escaping may be required

As you can see, the date format includes colon characters separating the hours, minutes, and seconds. Because the colon is a special character to Solr's most common query parsers, escaping is sometimes required, depending on exactly what you are trying to do.

This is normally an invalid query:

datefield:1972-05-20T17:33:18.772Z

These are valid queries:

datefield:1972-05-20T17\:33\:18.772Z datefield:"1972-05-20T17:33:18.772Z" datefield:[1972-05-20T17:33:18.772 TO *]

Date Range Formatting

Solr's DateRangeField supports the same point in time date syntax described above (with date math describe d below) and more to express date ranges. One class of examples is truncated dates, which represent the entire date span to the precision indicated. The other class uses the range syntax ([ TO ]). Here are some examples:

2000-11 – The entire month of November, 2000.

2000-11T13 – Likewise but for an hour of the day (1300 to before 1400, i.e. 1pm to 2pm).

-0009 – The year 10 BC. A 0 in the year position is 0 AD, and is also considered 1 BC.

[2000-11-01 TO 2014-12-01] – The specified date range at a day resolution.

[2014 TO 2014-12-01] – From the start of 2014 till the end of the first day of December.

[* TO 2014-12-01] – From the earliest representable time thru till the end of the day on 2014-12-01.

Limitations: The range syntax doesn't support embedded date math. If you specify a date instance supported by TrieDateField with date math truncating it, like NOW/DAY, you still get the first millisecond of that day, not the entire day's range. Exclusive ranges (using { & }) work in queries but not for indexing ranges.

Date Math

Solr's date field types also supports date math expressions, which makes it easy to create times relative to fixed moments in time, include the current time which can be represented using the special value of "NOW".

Date Math Syntax

Date math expressions consist either adding some quantity of time in a specified unit, or rounding the current time by a specified unit. expressions can be chained and are evaluated left to right.

For example: this represents a point in time two months from now:

NOW+2MONTHS

This is one day ago:

NOW-1DAY

A slash is used to indicate rounding. This represents the beginning of the current hour:

NOW/HOUR

The following example computes (with millisecond precision) the point in time six months and three days into the future and then rounds that time to the beginning of that day:

NOW+6MONTHS+3DAYS/DAY

Note that while date math is most commonly used relative to NOW it can be applied to any fixed moment in time as well:

1972-05-20T17:33:18.772Z+6MONTHS+3DAYS/DAY

Request Parameters That Affect Date Math

NOW

The NOW parameter is used internally by Solr to ensure consistent date math expression parsing across multiple nodes in a distributed request. But it can be specified to instruct Solr to use an arbitrary moment in time (past or future) to override for all situations where the the special value of "NOW" would impact date math expressions.

It must be specified as a (long valued) milliseconds since epoch Example:

q=solr&fq=start_date:[* TO NOW]&NOW=1384387200000

TZ

By default, all date math expressions are evaluated relative to the UTC TimeZone, but the TZ parameter can be specified to override this behaviour, by forcing all date based addition and rounding to be relative to the specified time zone.

For example, the following request will use range faceting to facet over the current month, "per day" relative UTC:

http://localhost:8983/solr/my_collection/select?q=*:*&facet.range=my_date_field&face t=true&facet.range.start=NOW/MONTH&facet.range.end=NOW/MONTH%2B1MONTH&facet.range.ga p=%2B1DAY

<int name="2013-11-01T00:00:00Z">0</int>

<int name="2013-11-02T00:00:00Z">0</int>

<int name="2013-11-03T00:00:00Z">0</int>

<int name="2013-11-04T00:00:00Z">0</int>

<int name="2013-11-05T00:00:00Z">0</int>

<int name="2013-11-06T00:00:00Z">0</int>

<int name="2013-11-07T00:00:00Z">0</int>

...

While in this example, the "days" will be computed relative to the specified time zone - including any applicable Daylight Savings Time adjustments:

http://localhost:8983/solr/my_collection/select?q=*:*&facet.range=my_date_field&face t=true&facet.range.start=NOW/MONTH&facet.range.end=NOW/MONTH%2B1MONTH&facet.range.ga p=%2B1DAY&TZ=America/Los_Angeles

<int name="2013-11-01T07:00:00Z">0</int>

<int name="2013-11-02T07:00:00Z">0</int>

<int name="2013-11-03T07:00:00Z">0</int>

<int name="2013-11-04T08:00:00Z">0</int>

<int name="2013-11-05T08:00:00Z">0</int>

<int name="2013-11-06T08:00:00Z">0</int>

<int name="2013-11-07T08:00:00Z">0</int>

...

More DateRangeField Details

DateRangeField is almost a drop-in replacement for places where TrieDateField is used. The only difference is that Solr's XML or SolrJ response formats will expose the stored data as a String instead of a Date. The underlying index data for this field will be a bit larger. Queries that align to units of time a second on up should be faster than TrieDateField, especially if it's in UTC. But the main point of DateRangeField as its name suggests is to allow indexing date ranges. To do that, simply supply strings in the format shown above. It also supports specifying 3 different relational predicates between the indexed data, and the query range: Intersect s (default), Contains, Within. You can specify the predicate by querying using the op local-params parameter like so:

fq={!field f=dateRange op=Contains}[2013 TO 2018]

Unlike most/all local-params, op is actually not defined by any query parser (field), it is defined by the field type – DateRangeField. In that example, it would find documents with indexed ranges that contain (or equals) the range 2013 thru 2018. Multi-valued overlapping indexed ranges in a document are effectively coalesced.

For a DateRangeField example use-case and possibly other information, see Solr's community wiki.

Working with Enum Fields‌

The EnumField type allows defining a field whose values are a closed set, and the sort order is pre-determined but is not alphabetic nor numeric. Examples of this are severity lists, or risk definitions.

Defining an EnumField in schema.xml

The EnumField type definition is quite simple, as in this example defining field types for "priorityLevel" and "riskLevel" enumerations:

<fieldType name="priorityLevel" class="solr.EnumField" enumsConfig="enumsConfig.xml" enumName="priority"/>

<fieldType name="riskLevel" class="solr.EnumField" enumsConfig="enumsConfig.xml" enumName="risk" />

Besides the name and the class, which are common to all field types, this type also takes two additional parameters:

enumsConfig: the name of a configuration file that contains the <enum/> list of field values and their order that you wish to use with this field type. If a path to the file is not defined specified, the file should be in the conf directory for the collection.

enumName: the name of the specific enumeration in the enumsConfig file to use for this type.

Defining the EnumField configuration file

The file named with the enumsConfig parameter can contain multiple enumeration value lists with different names if there are multiple uses for enumerations in your Solr schema.

In this example, there are two value lists defined. Each list is between enum opening and closing tags:

<?xml version="1.0" ?>

<enumsConfig>

<enum name="priority">

<value>Not Available</value>

<value>Low</value>

<value>Medium</value>

<value>High</value>

<value>Urgent</value>

</enum>

<enum name="risk">

<value>Unknown</value>

<value>Very Low</value>

<value>Low</value>

<value>Medium</value>

<value>High</value>

<value>Critical</value>

</enum>

</enumsConfig>

Changing Values

You cannot change the order, or remove, existing values in an <enum/> without reindexing. You can however add new values to the end.

Working with External Files and Processes‌

The ExternalFileField Type Format of the External File Reloading an External File

The PreAnalyzedField Type JsonPreAnalyzedParser SimplePreAnalyzedParser

The ExternalFileField Type

The ExternalFileField type makes it possible to specify the values for a field in a file outside the Solr index. For such a field, the file contains mappings from a key field to the field value. Another way to think of this is that, instead of specifying the field in documents as they are indexed, Solr finds values for this field in the external file.

External fields are not searchable. They can be used only for function queries or display. For more information on function queries, see the section on Function Queries.

The ExternalFileField type is handy for cases where you want to update a particular field in many documents more often than you want to update the rest of the documents. For example, suppose you have implemented a document rank based on the number of views. You might want to update the rank of all the documents daily or hourly, while the rest of the contents of the documents might be updated much less frequently. Without ExternalFileField, you would need to update each document just to change the rank. Using ExternalFileField is much more efficient because all document values for a particular field are stored in an external file that can be updated as frequently as you wish.

In schema.xml, the definition of this field type might look like this:

<fieldType name="entryRankFile" keyField="pkId" defVal="0" stored="false" indexed="false" class="solr.ExternalFileField" valType="pfloat"/>

The keyField attribute defines the key that will be defined in the external file. It is usually the unique key for the index, but it doesn't need to be as long as the keyField can be used to identify documents in the index. A defV al defines a default value that will be used if there is no entry in the external file for a particular document.

The valType attribute specifies the actual type of values that will be found in the file. The type specified must be either a float field type, so valid values for this attribute are pfloat, float or tfloat. This attribute can be omitted.

Format of the External File

The file itself is located in Solr's index directory, which by default is $SOLR_HOME/data. The name of the file should be external_fieldname or external_fieldname.*. For the example above, then, the file could be named external_entryRankFile or external_entryRankFile.txt.

If any files using the name pattern .* (such as .txt) appear, the last (after being sorted by name) will be used and previous versions will be deleted. This behavior supports implementations on systems where one may not be able to overwrite a file (for example, on Windows, if the file is in use).

The file contains entries that map a key field, on the left of the equals sign, to a value, on the right. Here are a few example entries:

doc33=1.414 doc34=3.14159

doc40=42

The keys listed in this file do not need to be unique. The file does not need to be sorted, but Solr will be able to perform the lookup faster if it is.

Reloading an External File

It's possible to define an event listener to reload an external file when either a searcher is reloaded or when a new searcher is started. See the section Query-Related Listeners for more information, but a sample definition in solrconfig.xml might look like this:

<listener event="newSearcher" class="org.apache.solr.schema.ExternalFileFieldReloader"/>

<listener event="firstSearcher" class="org.apache.solr.schema.ExternalFileFieldReloader"/>

The PreAnalyzedField Type

The PreAnalyzedField type provides a way to send to Solr serialized token streams, optionally with independent stored values of a field, and have this information stored and indexed without any additional text processing applied in Solr. This is useful if user wants to submit field content that was already processed by some existing external text processing pipeline (e.g., it has been tokenized, annotated, stemmed, synonyms inserted, etc.), while using all the rich attributes that Lucene's TokenStream provides (per-token attributes).

The serialization format is pluggable using implementations of PreAnalyzedParser interface. There are two out-of-the-box implementations:

JsonPreAnalyzedParser: as the name suggests, it parses content that uses JSON to represent field's content. This is the default parser to use if the field type is not configured otherwise.

SimplePreAnalyzedParser: uses a simple strict plain text format, which in some situations may be easier to create than JSON.

There is only one configuration parameter, parserImpl. The value of this parameter should be a fully qualified class name of a class that implements PreAnalyzedParser interface. The default value of this parameter is org. apache.solr.schema.JsonPreAnalyzedParser.

By default, the query-time analyzer for fields of this type will be the same as the index-time analyzer, which expects serialized pre-analyzed text. You must add a query type analyzer to your fieldType in order to perform analysis on non-pre-analyzed queries. In the example below, the index-time analyzer expects the default JSON serialization format, and the query-time analyzer will employ StandardTokenizer/LowerCaseFilter:

<fieldType name="pre_with_query_analyzer" class="solr.PreAnalyzedField">

<analyzer type="query">

<tokenizer class="solr.StandardTokenizerFactory"/>

<filter class="solr.LowerCaseFilterFactory"/>

</analyzer>

</fieldType>

JsonPreAnalyzedParser

This is the default serialization format used by PreAnalyzedField type. It uses a top-level JSON map with the following keys:

Any other top-level key is silently ignored.

Token stream serialization

The token stream is expressed as a JSON list of JSON maps. The map for each token consists of the following keys and values:

Any other key is silently ignored.

Example

{

"v":"1",

"str":"test ó", "tokens": [

{"t":"one","s":123,"e":128,"i":22,"p":"DQ4KDQsODg8=","y":"word"},

{"t":"two","s":5,"e":8,"i":1,"y":"word"},

{"t":"three","s":20,"e":22,"i":1,"y":"foobar"}

]

}

SimplePreAnalyzedParser

The fully qualified class name to use when specifying this format via the parserImpl configuration parameter is

org.apache.solr.schema.SimplePreAnalyzedParser.

Syntax

The serialization format supported by this parser is as follows:

Serialization format

content ::= version (stored)? tokens version ::= digit+ " "

; stored field value - any "=" inside must be escaped! stored ::= "=" text "="

tokens ::= (token ((" ") + token)*)* token ::= text ("," attrib)*

attrib ::= name '=' value name ::= text

value ::= text

Special characters in "text" values can be escaped using the escape character \ . The following escape sequences are recognized:

Please note that Unicode sequences (e.g. \u0001) are not supported.

Supported attribute names

The following token attributes are supported, and identified with short symbolic names:

Token positions are tracked and implicitly added to the token stream - the start and end offsets consider only the term text and whitespace, and exclude the space taken by token attributes.

Example token streams

1 one two three

version: 1 stored: null

token: (term=one,startOffset=0,endOffset=3) token: (term=two,startOffset=4,endOffset=7) token: (term=three,startOffset=8,endOffset=13)

1 one two three

version: 1 stored: null

token: (term=one,startOffset=0,endOffset=3) token: (term=two,startOffset=5,endOffset=8) token: (term=three,startOffset=11,endOffset=16)

1 one,s=123,e=128,i=22 two three,s=20,e=22

version: 1 stored: null

token: (term=one,positionIncrement=22,startOffset=123,endOffset=128) token: (term=two,positionIncrement=1,startOffset=5,endOffset=8) token: (term=three,positionIncrement=1,startOffset=20,endOffset=22)

1 \ one\ \,,i=22,a=\, two\=

\n,\ =\ \

version: 1 stored: null

token: (term= one ,,positionIncrement=22,startOffset=0,endOffset=6) token: (term=two=

,positionIncrement=1,startOffset=7,endOffset=15)

token: (term=\,positionIncrement=1,startOffset=17,endOffset=18)

Note that unknown attributes and their values are ignored, so in this example, the "a" attribute on the first token and the " " (escaped space) attribute on the second token are ignored, along with their values, because they are not among the supported attribute names.

1 ,i=22 ,i=33,s=2,e=20 ,

version: 1 stored: null

token: (term=,positionIncrement=22,startOffset=0,endOffset=0) token: (term=,positionIncrement=33,startOffset=2,endOffset=20) token: (term=,positionIncrement=1,startOffset=2,endOffset=2)

1 =This is the stored part with \=

\n \t escapes.=one two three

version: 1

stored: "This is the stored part with =

\t escapes."

token: (term=one,startOffset=0,endOffset=3) token: (term=two,startOffset=4,endOffset=7) token: (term=three,startOffset=8,endOffset=13)

Note that the "\t" in the above stored value is not literal; it's shown that way to visually indicate the actual tab char that is in the stored value.

1 ==

version: 1 stored: "" (no tokens)

1 =this is a test.=

version: 1

stored: "this is a test." (no tokens)

Field Properties by Use Case‌

Here is a summary of common use cases, and the attributes the fields or field types should have to support the case. An entry of true or false in the table indicates that the option must be set to the given value for the use case to function correctly. If no entry is provided, the setting of that attribute has no impact on the case.

Notes:

1. Recommended but not necessary.

2. Will be used if present, but not necessary.

3. (if termVectors=true)

4. A tokenizer must be defined for the field, but it doesn't need to be indexed.

5. Described in Understanding Analyzers, Tokenizers, and Filters.

6. Term vectors are not mandatory here. If not true, then a stored field is analyzed. So term vectors are recommended, but only required if stored=false.

7. Either indexed or docValues must be true, but both are not required. DocValues can be more efficient in many cases.

   
   

   Defining Fields‌

   Fields are defined in the fields element of schema.xml. Once you have the field types set up, defining the fields themselves is simple.

   
   

   Example

   
   

   The following example defines a field named price with a type named float and a default value of 0.0; the i ndexed and stored properties are explicitly set to true, while any other properties specified on the float fiel d type are inherited.

   
   

   
   

   <field name="price" type="float" default="0.0" indexed="true" stored="true"/>

   
   

   Field Properties

   
   

   Property	Description
   name	The name of the field. Field names should consist of alphanumeric or underscore characters only and not start with a digit. This is not currently strictly enforced, but other field names will not have first class support from all components and back compatibility is not guaranteed. Names with both leading and trailing underscores (e.g. _version_) are reserved. Every field must have a name.
   type	The name of the fieldType for this field. This will be found in the "name" attribute on the field Type definition. Every field must have a type.

   
   

   

   default A default value that will be added automatically to any document that does not have a value in this field when it is indexed. If this property is not specified, there is no default.

   
   

   Optional Field Type Override Properties

   
   

   Fields can have many of the same properties as field types. Properties from the table below which are specified on an individual field will override any explicit value for that property specified on the the <fieldType/> of the field, or any implicit default property value provided by the underlying FieldType implementation. The table below is reproduced from Field Type Definitions and Properties, which has more details:

   
   

   Property	Description	Values	Implicit Default
   indexed	If true, the value of the field can be used in queries to retrieve matching documents.	true or false	true
   stored	If true, the actual value of the field can be retrieved by queries.	true or false	true
   docValues	If true, the value of the field will be put in a column-oriented DocValues structure.	true or false	false
   sortMissingFirst sortMissingLast	Control the placement of documents when a sort field is not present.	true or false	false
   multiValued	If true, indicates that a single document might contain multiple values for this field type.	true or false	false
   omitNorms	If true, omits the norms associated with this field (this disables length normalization and index-time boosting for the field, and saves some memory). Defaults to true for all primitive (non-analyzed) field types, such as int, float, data, bool, and string. Only full-text fields or fields that need an index-time boost need norms.	true or false	*
   omitTermFreqAndPositions	If true, omits term frequency, positions, and payloads from postings for this field. This can be a performance boost for fields that don't require that information. It also reduces the storage space required for the index. Queries that rely on position that are issued on a field with this option will silently fail to find documents. This property defaults to true for all field types that are not text fields.	true or false	*
   omitPositions	Similar to omitTermFreqAndPositions but preserves term frequency information.	true or false	*
   termVectors termPositions termOffsets termPayloads	These options instruct Solr to maintain full term vectors for each document, optionally including position, offset and payload information for each term occurrence in those vectors. These can be used to accelerate highlighting and other ancillary functionality, but impose a substantial cost in terms of index size. They are not necessary for typical uses of Solr.	true or false	false
   required	Instructs Solr to reject any attempts to add a document which does not have a value for this field. This property defaults to false.	true or false	false

   
   

   useDocValuesAsStored

   If the field has docValues enabled, setting this to true would allow the field to be returned as if it were a stored field (even if it has stored=false) when matching "*" in an fl parameter.

   true or false

   true

   
   

   Related Topics‌

   
   

   

   SchemaXML-Fields

   

   Field Options by Use Case

   
   

   Copying Fields‌

   You might want to interpret some document fields in more than one way. Solr has a mechanism for making copies of fields so that you can apply several distinct field types to a single piece of incoming information.

   The name of the field you want to copy is the source, and the name of the copy is the destination. In schema.xm l, it's very simple to make copies of fields:

   
   

   
   

   <copyField source="cat" dest="text" maxChars="30000" />

   
   

   In this example, we want Solr to copy the cat field to a field named text. Fields are copied before analysis is done, meaning you can have two fields with identical original content, but which use different analysis chains and are stored in the index differently.

   In the example above, if the text destination field has data of its own in the input documents, the contents of the cat field will be added as additional values – just as if all of the values had originally been specified by the client. Remember to configure your fields as multivalued="true" if they will ultimately get multiple values (either from a multivalued source or from multiple copyField directives).

   A common usage for this functionality is to create a single "search" field that will serve as the default query field when users or clients do not specify a field to query. For example, title, author, keywords, and body may all be fields that should be searched by default, with copy field rules for each field to copy to a catchall field (for example, it could be named anything). Later you can set a rule in solrconfig.xml to search the catchal l field by default. One caveat to this is your index will grow when using copy fields. However, whether this becomes problematic for you and the final size will depend on the number of fields being copied, the number of destination fields being copied to, the analysis in use, and the available disk space.

   The maxChars parameter, an int parameter, establishes an upper limit for the number of characters to be copied from the source value when constructing the value added to the destination field. This limit is useful for situations in which you want to copy some data from the source field, but also control the size of index files.

   

   Both the source and the destination of copyField can contain either leading or trailing asterisks, which will match anything. For example, the following line will copy the contents of all incoming fields that match the wildcard pattern *_t to the text field.:

   
   

   
   

   <copyField source="*_t" dest="text" maxChars="25000" />

   
   

   The copyField command can use a wildcard (*) character in the dest parameter only if the source p arameter contains one as well. copyField uses the matching glob from the source field for the dest fie ld name into which the source content is copied.

   Dynamic Fields

   Dynamic fields allow Solr to index fields that you did not explicitly define in your schema. This is useful if you discover you have forgotten to define one or more fields. Dynamic fields can make your application less brittle by providing some flexibility in the documents you can add to Solr.

   A dynamic field is just like a regular field except it has a name with a wildcard in it. When you are indexing documents, a field that does not match any explicitly defined fields can be matched with a dynamic field.

   For example, suppose your schema includes a dynamic field with a name of *_i. If you attempt to index a document with a cost_i field, but no explicit cost_i field is defined in the schema, then the cost_i field will have the field type and analysis defined for *_i.

   Like regular fields, dynamic fields have a name, a field type, and options.

   
   

   
   

   <dynamicField name="*_i" type="int" indexed="true" stored="true"/>

   
   

   It is recommended that you include basic dynamic field mappings (like that shown above) in your schema.xml. The mappings can be very useful.

   
   

   Related Topics

   
   

   

   SchemaXML-Dynamic Fields

   
   

   Other Schema Elements‌

   This section describes several other important elements of schema.xml.

   
   

   Unique Key

   
   

   The uniqueKey element specifies which field is a unique identifier for documents. Although uniqueKey is not required, it is nearly always warranted by your application design. For example, uniqueKey should be used if you will ever update a document in the index.

   You can define the unique key field by naming it:

   
   

   
   

   <uniqueKey>id</uniqueKey>

   
   

   Schema defaults and copyFields cannot be used to populate the uniqueKey field. The fieldType of uniqu eKey must not be analyzed. You can use UUIDUpdateProcessorFactory to have uniqueKey values generated automatically.

   Further, the operation will fail if the uniqueKey field is used, but is multivalued (or inherits the multivalueness from the fieldtype). However, uniqueKey will continue to work, as long as the field is properly used.

   
   

   Default Search Field & Query Operator

   
   

   Although they have been deprecated for quite some time, Solr still has support for Schema based configuration of a <defaultSearchField/> (which is superseded by the df parameter) and <solrQueryParser defaultOperator="OR"/> (which is superseded by the q.op parameter.

   If you have these options specified in your Schema, you are strongly encouraged to replace them with request

   parameters (or request parameter defaults) as support for them may be removed from future Solr release.

   
   

   Similarity

   
   

   Similarity is a Lucene class used to score a document in searching.

   Each collection has one "global" Similarity, and by default Solr uses an implicit SchemaSimilarityFactory w hich allows individual field types to be configured with a "per-type" specific Similarity and implicitly uses BM25Sim ilarity for any field type which does not have an explicit Similarity.

   This default behavior can be overridden by declaring a top level <similarity/> element in your schema.xml, outside of any single field type. This similarity declaration can either refer directly to the name of a class with a no-argument constructor, such as in this example showing BM25Similarity:

   
   

   
   

   <similarity class="solr.BM25SimilarityFactory"/>

   
   

   or by referencing a SimilarityFactory implementation, which may take optional initialization parameters:

   
   

   
   

   <similarity class="solr.DFRSimilarityFactory">

   <str name="basicModel">P</str>

   <str name="afterEffect">L</str>

   <str name="normalization">H2</str>

   <float name="c">7</float>

   </similarity>

   
   

   In most cases, specifying global level similarity like this will cause an error if your schema.xml also includes field type specific <similarity/> declarations. One key exception to this is that you may explicitly declare a S chemaSimilarityFactory and specify what that default behavior will be for all field types that do not declare an explicit Similarity using the name of field type (specified by defaultSimFromFieldType) that is configured with a specific similarity:

   
   

   
   

   <similarity class="solr.SchemaSimilarityFactory">

   <str name="defaultSimFromFieldType">text_dfr</str>

   </similarity>

   <fieldType name="text_dfr" class="solr.TextField">

   <analyzer ... />

   <similarity class="solr.DFRSimilarityFactory">

   <str name="basicModel">I(F)</str>

   <str name="afterEffect">B</str>

   <str name="normalization">H3</str>

   <float name="mu">900</float>

   </similarity>

   </fieldType>

   <fieldType name="text_ib" class="solr.TextField">

   <analyzer ... />

   <similarity class="solr.IBSimilarityFactory">

   <str name="distribution">SPL</str>

   <str name="lambda">DF</str>

   <str name="normalization">H2</str>

   </similarity>

   </fieldType>

   <fieldType name="text_other" class="solr.TextField">

   <analyzer ... />

   </fieldType>

   In the example above IBSimilarityFactory (using the Information-Based model) will be used for any fields of type text_ib, while DFRSimilarityFactory (divergence from random) will be used for any fields of type t ext_dfr, as well as any fields using a type that does not explicitly specify a <similarity/>.

   If SchemaSimilarityFactory is explicitly declared with out configuring a defaultSimFromFieldType, then BM25Similarity is implicitly used as the default.

   In addition to the various factories mentioned on this page, there are several other similarity implementations that can be used such as the SweetSpotSimilarityFactory, ClassicSimilarityFactory, etc.... For details, see the Solr Javadocs for the similarity factories.

   
   

   Schema API‌

   

   The Schema API provides read and write access to the Solr schema for each collection (or core, when using standalone Solr). Read access to all schema elements is supported. Fields, dynamic fields, field types and copyField rules may be added, removed or replaced. Future Solr releases will extend write access to allow more schema elements to be modified.

   
   

   Re-index after schema modifications!

   If you modify your schema, you will likely need to re-index all documents. If you do not, you may lose access to documents, or not be able to interpret them properly, e.g. after replacing a field type.

   Modifying your schema will never modify any documents that are already indexed. Again, you must re-index documents in order to apply schema changes to them.

   
   

   To enable schema modification with this API, the schema will need to be managed and mutable. See the section Schema Factory Definition in SolrConfig for more information.

   The API allows two output modes for all calls: JSON or XML. When requesting the complete schema, there is another output mode which is XML modeled after the schema.xml file itself.

   When modifying the schema with the API, a core reload will automatically occur in order for the changes to be available immediately for documents indexed thereafter. Previously indexed documents will not be automatically handled - they must be re-indexed if they used schema elements that you changed.

   The base address for the API is http://<host>:<port>/solr/<collection_name>. If for example you run Solr's "cloud" example (via the bin/solr command shown below), which creates a "gettingstarted" collection, then the base URL (as in all the sample URLs in this section) would be: http://localhost:8983/ solr/gettingstarted .

   
   

   
   

   bin/solr -e cloud -noprompt

   
   

   

   

   API Entry Points Modify the Schema

   

   

   

   Add a New Field Delete a Field Replace a Field

   

   

   

   

   Add a Dynamic Field Rule Delete a Dynamic Field Rule Replace a Dynamic Field Rule Add a New Field Type

   

   

   Delete a Field Type Replace a Field Type

   

   Add a New Copy Field Rule

   

   Delete a Copy Field Rule

   

   

   Multiple Commands in a Single POST Schema Changes among Replicas

   

   

   

   Retrieve Schema Information Retrieve the Entire Schema List Fields

   

   

   

   List Dynamic Fields List Field Types List Copy Fields

   

   

   

   Show Schema Name Show the Schema Version List UniqueKey

   

   Show Global Similarity

   

   

   Get the Default Query Operator Manage Resource Data

   
   

   API Entry Points

   
   

   /schema: retrieve the schema, or modify the schema to add, remove, or replace fields, dynamic fields, copy fields, or field types

   /schema/fields: retrieve information about all defined fields or a specific named field

   /schema/dynamicfields: retrieve information about all dynamic field rules or a specific named dynamic rule

   /schema/fieldtypes: retrieve information about all field types or a specific field type

   /schema/copyfields: retrieve information about copy fields

   /schema/name: retrieve the schema name

   /schema/version: retrieve the schema version

   /schema/uniquekey: retrieve the defined uniqueKey

   /schema/similarity: retrieve the global similarity definition

   /schema/solrqueryparser/defaultoperator: retrieve the default operator

   
   

   Modify the Schema

   
   

   POST /collection/schema

   To add, remove or replace fields, dynamic field rules, copy field rules, or new field types, you can send a POST request to the /collection/schema/ endpoint with a sequence of commands to perform the requested actions. The following commands are supported:

   

   add-field: add a new field with parameters you provide.

   

   delete-field: delete a field.

   

   replace-field: replace an existing field with one that is differently configured.

   

   add-dynamic-field: add a new dynamic field rule with parameters you provide.

   

   delete-dynamic-field: delete a dynamic field rule.

   

   replace-dynamic-field: replace an existing dynamic field rule with one that is differently configured.

   

   add-field-type: add a new field type with parameters you provide.

   

   delete-field-type: delete a field type.

   

   replace-field-type: replace an existing field type with one that is differently configured.

   

   add-copy-field: add a new copy field rule.

   

   delete-copy-field: delete a copy field rule.

   These commands can be issued in separate POST requests or in the same POST request. Commands are executed in the order in which they are specified.

   In each case, the response will include the status and the time to process the request, but will not include the

   entire schema.

   When modifying the schema with the API, a core reload will automatically occur in order for the changes to be available immediately for documents indexed thereafter. Previously indexed documents will not be automatically handled - they must be re-indexed if they used schema elements that you changed.

   
   

   Add a New Field

   
   

   The add-field command adds a new field definition to your schema. If a field with the same name exists an error is thrown.

   All of the properties available when defining a field with manual schema.xml edits can be passed via the API. These request attributes are described in detail in the section Defining Fields.

   For example, to define a new stored field named "sell-by", of type "tdate", you would POST the following request:

   
   

   
   

   curl -X POST -H 'Content-type:application/json' --data-binary '{ "add-field":{

   "name":"sell-by",

   "type":"tdate", "stored":true }

   }' http://localhost:8983/solr/gettingstarted/schema

   
   

   Delete a Field

   
   

   The delete-field command removes a field definition from your schema. If the field does not exist in the schema, or if the field is the source or destination of a copy field rule, an error is thrown.

   For example, to delete a field named "sell-by", you would POST the following request:

   
   

   
   

   curl -X POST -H 'Content-type:application/json' --data-binary '{ "delete-field" : { "name":"sell-by" }

   }' http://localhost:8983/solr/gettingstarted/schema

   
   

   Replace a Field

   
   

   The replace-field command replaces a field's definition. Note that you must supply the full definition for a field - this command will not partially modify a field's definition. If the field does not exist in the schema an error is thrown.

   All of the properties available when defining a field with manual schema.xml edits can be passed via the API. These request attributes are described in detail in the section Defining Fields.

   For example, to replace the definition of an existing field "sell-by", to make it be of type "date" and to not be stored, you would POST the following request:

   
   

   
   

   curl -X POST -H 'Content-type:application/json' --data-binary '{ "replace-field":{

   "name":"sell-by",

   "type":"date", "stored":false }

   }' http://localhost:8983/solr/gettingstarted/schema

   Add a Dynamic Field Rule

   
   

   The add-dynamic-field command adds a new dynamic field rule to your schema.

   All of the properties available when editing schema.xml can be passed with the POST request. The section Dyn amic Fields has details on all of the attributes that can be defined for a dynamic field rule.

   For example, to create a new dynamic field rule where all incoming fields ending with "_s" would be stored and have field type "string", you can POST a request like this:

   
   

   
   

   curl -X POST -H 'Content-type:application/json' --data-binary '{ "add-dynamic-field":{

   "name":"*_s",

   "type":"string", "stored":true }

   }' http://localhost:8983/solr/gettingstarted/schema

   
   

   Delete a Dynamic Field Rule

   
   

   The delete-dynamic-field command deletes a dynamic field rule from your schema. If the dynamic field rule does not exist in the schema, or if the schema contains a copy field rule with a target or destination that matches only this dynamic field rule, an error is thrown.

   For example, to delete a dynamic field rule matching "*_s", you can POST a request like this:

   
   

   
   

   curl -X POST -H 'Content-type:application/json' --data-binary '{ "delete-dynamic-field":{ "name":"*_s" }

   }' http://localhost:8983/solr/gettingstarted/schema

   
   

   Replace a Dynamic Field Rule

   
   

   The replace-dynamic-field command replaces a dynamic field rule in your schema. Note that you must supply the full definition for a dynamic field rule - this command will not partially modify a dynamic field rule's definition. If the dynamic field rule does not exist in the schema an error is thrown.

   All of the properties available when editing schema.xml can be passed with the POST request. The section Dyn amic Fields has details on all of the attributes that can be defined for a dynamic field rule.

   For example, to replace the definition of the "*_s" dynamic field rule with one where the field type is "text_general" and it's not stored, you can POST a request like this:

   
   

   
   

   curl -X POST -H 'Content-type:application/json' --data-binary '{ "replace-dynamic-field":{

   "name":"*_s", "type":"text_general", "stored":false }

   }' http://localhost:8983/solr/gettingstarted/schema

   
   

   Add a New Field Type

   
   

   The add-field-type command adds a new field type to your schema.

   All of the field type properties available when editing schema.xml by hand are available for use in a POST

   request. The structure of the command is a json mapping of the standard field type definition, including the name, class, index and query analyzer definitions, etc. Details of all of the available options are described in the section Solr Field Types.

   For example, to create a new field type named "myNewTxtField", you can POST a request as follows:

   
   

   
   

   curl -X POST -H 'Content-type:application/json' --data-binary '{ "add-field-type" : {

   "name":"myNewTxtField", "class":"solr.TextField", "positionIncrementGap":"100", "analyzer" : {

   "charFilters":[{ "class":"solr.PatternReplaceCharFilterFactory", "replacement":"$1$1",

   "pattern":"([a-zA-Z])\\\\1+" }], "tokenizer":{

   "class":"solr.WhitespaceTokenizerFactory" }, "filters":[{

   "class":"solr.WordDelimiterFilterFactory", "preserveOriginal":"0" }]}}

   }' http://localhost:8983/solr/gettingstarted/schema

   
   

   Note in this example that we have only defined a single analyzer section that will apply to index analysis and query analysis. If we wanted to define separate analysis, we would replace the analyzer section in the above example with separate sections for indexAnalyzer and queryAnalyzer. As in this example:

   
   

   
   

   curl -X POST -H 'Content-type:application/json' --data-binary '{ "add-field-type":{

   "name":"myNewTextField", "class":"solr.TextField", "indexAnalyzer":{

   "tokenizer":{ "class":"solr.PathHierarchyTokenizerFactory", "delimiter":"/" }},

   "queryAnalyzer":{ "tokenizer":{

   "class":"solr.KeywordTokenizerFactory" }}}

   }' http://localhost:8983/solr/gettingstarted/schema

   
   

   Delete a Field Type

   
   

   The delete-field-type command removes a field type from your schema. If the field type does not exist in the schema, or if any field or dynamic field rule in the schema uses the field type, an error is thrown.

   For example, to delete the field type named "myNewTxtField", you can make a POST request as follows:

   
   

   
   

   curl -X POST -H 'Content-type:application/json' --data-binary '{ "delete-field-type":{ "name":"myNewTxtField" }

   }' http://localhost:8983/solr/gettingstarted/schema

   
   

   Replace a Field Type

   
   

   The replace-field-type command replaces a field type in your schema. Note that you must supply the full

   definition for a field type - this command will not partially modify a field type's definition. If the field type does not exist in the schema an error is thrown.

   All of the field type properties available when editing schema.xml by hand are available for use in a POST request. The structure of the command is a json mapping of the standard field type definition, including the name, class, index and query analyzer definitions, etc. Details of all of the available options are described in the section Solr Field Types.

   For example, to replace the definition of a field type named "myNewTxtField", you can make a POST request as follows:

   
   

   
   

   curl -X POST -H 'Content-type:application/json' --data-binary '{ "replace-field-type":{

   "name":"myNewTxtField", "class":"solr.TextField", "positionIncrementGap":"100", "analyzer":{

   "tokenizer":{ "class":"solr.StandardTokenizerFactory" }}}

   }' http://localhost:8983/solr/gettingstarted/schema

   
   

   Add a New Copy Field Rule

   
   

   The add-copy-field command adds a new copy field rule to your schema.

   The attributes supported by the command are the same as when creating copy field rules by manually editing the schema.xml, as below:

   
   

   Name	Required	Description
   source	Yes	The source field.
   dest	Yes	A field or an array of fields to which the source field will be copied.
   maxChars	No	The upper limit for the number of characters to be copied. The section Copying Fields has more details.

   
   

   For example, to define a rule to copy the field "shelf" to the "location" and "catchall" fields, you would POST the following request:

   
   

   
   

   curl -X POST -H 'Content-type:application/json' --data-binary '{ "add-copy-field":{

   "source":"shelf",

   "dest":[ "location", "catchall" ]}

   }' http://localhost:8983/solr/gettingstarted/schema

   
   

   Delete a Copy Field Rule

   
   

   The delete-copy-field command deletes a copy field rule from your schema. If the copy field rule does not exist in the schema an error is thrown.

   The source and dest attributes are required by this command.

   For example, to delete a rule to copy the field "shelf" to the "location" field, you would POST the following request:

   
   

   
   

   curl -X POST -H 'Content-type:application/json' --data-binary '{ "delete-copy-field":{ "source":"shelf", "dest":"location" }

   }' http://localhost:8983/solr/gettingstarted/schema

   
   

   Multiple Commands in a Single POST

   
   

   It is possible to perform one or more add requests in a single command. The API is transactional and all commands in a single call either succeed or fail together.

   The commands are executed in the order in which they are specified. This means that if you want to create a new field type and in the same request use the field type on a new field, the section of the request that creates the field type must come before the section that creates the new field. Similarly, since a field must exist for it to be used in a copy field rule, a request to add a field must come before a request for the field to be used as either the source or the destination for a copy field rule.

   The syntax for making multiple requests supports several approaches. First, the commands can simply be made serially, as in this request to create a new field type and then a field that uses that type:

   
   

   
   

   curl -X POST -H 'Content-type:application/json' --data-binary '{ "add-field-type":{

   "name":"myNewTxtField", "class":"solr.TextField", "positionIncrementGap":"100", "analyzer":{

   "charFilters":[{ "class":"solr.PatternReplaceCharFilterFactory", "replacement":"$1$1",

   "pattern":"([a-zA-Z])\\\\1+" }], "tokenizer":{

   "class":"solr.WhitespaceTokenizerFactory" }, "filters":[{

   "class":"solr.WordDelimiterFilterFactory", "preserveOriginal":"0" }]}},

   "add-field" : {

   "name":"sell-by", "type":"myNewTxtField", "stored":true }

   }' http://localhost:8983/solr/gettingstarted/schema

   
   

   Or, the same command can be repeated, as in this example:

   
   

   
   

   curl -X POST -H 'Content-type:application/json' --data-binary '{ "add-field":{

   "name":"shelf", "type":"myNewTxtField", "stored":true },

   "add-field":{

   "name":"location", "type":"myNewTxtField", "stored":true },

   "add-copy-field":{

   "source":"shelf",

   "dest":[ "location", "catchall" ]}

   }' http://localhost:8983/solr/gettingstarted/schema

   Finally, repeated commands can be sent as an array:

   
   

   
   

   curl -X POST -H 'Content-type:application/json' --data-binary '{ "add-field":[

   { "name":"shelf", "type":"myNewTxtField", "stored":true },

   { "name":"location", "type":"myNewTxtField", "stored":true }]

   }' http://localhost:8983/solr/gettingstarted/schema

   
   

   Schema Changes among Replicas

   
   

   When running in SolrCloud mode, changes made to the schema on one node will propagate to all replicas in the collection. You can pass the updateTimeoutSecs parameter with your request to set the number of seconds to wait until all replicas confirm they applied the schema updates. This helps your client application be more robust in that you can be sure that all replicas have a given schema change within a defined amount of time. If agreement is not reached by all replicas in the specified time, then the request fails and the error message will include information about which replicas had trouble. In most cases, the only option is to re-try the change after waiting a brief amount of time. If the problem persists, then you'll likely need to investigate the server logs on the replicas that had trouble applying the changes. If you do not supply an updateTimeoutSecs parameter, the default behavior is for the receiving node to return immediately after persisting the updates to ZooKeeper. All other replicas will apply the updates asynchronously. Consequently, without supplying a timeout, your client application cannot be sure that all replicas have applied the changes.

   
   

   Retrieve Schema Information

   
   

   The following endpoints allow you to read how your schema has been defined. You can GET the entire schema, or only portions of it as needed.

   To modify the schema, see the previous section Modify the Schema.

   
   

   Retrieve the Entire Schema

   
   

   GET /collection/schema

   
   

   INPUT

   Path Parameters

   
   

   Key	Description
   collection	The collection (or core) name.

   
   

   Query Parameters

   The query parameters should be added to the API request after '?'.

   
   

   Key	Type	Required	Default	Description
   wt	string	No	json	Defines the format of the response. The options are json, xml or schem a.xml. If not specified, JSON will be returned by default.

   
   

   OUTPUT

   Output Content

   The output will include all fields, field types, dynamic rules and copy field rules, in the format requested (JSON or XML). The schema name and version are also included.

   
   

   EXAMPLES

   Get the entire schema in JSON.

   
   

   
   

   curl http://localhost:8983/solr/gettingstarted/schema?wt=json

   

   
   

   {

   "responseHeader":{ "status":0,

   "QTime":5},

   "schema":{ "name":"example",

   "version":1.5,

   "uniqueKey":"id",

   "fieldTypes":[{ "name":"alphaOnlySort", "class":"solr.TextField", "sortMissingLast":true, "omitNorms":true, "analyzer":{

   "tokenizer":{ "class":"solr.KeywordTokenizerFactory"},

   "filters":[{

   "class":"solr.LowerCaseFilterFactory"},

   
   

   ...

   {

   "class":"solr.TrimFilterFactory"},

   {

   "class":"solr.PatternReplaceFilterFactory", "replace":"all",

   "replacement":"", "pattern":"([^a-z])"}]}},

   "fields":[{

   "name":"_version_",

   "type":"long", "indexed":true, "stored":true},

   
   

   ...

   {

   "name":"author", "type":"text_general", "indexed":true, "stored":true},

   {

   "name":"cat",

   "type":"string", "multiValued":true, "indexed":true, "stored":true},

   "copyFields":[{

   "source":"author",

   "dest":"text"},

   
   

   ...

   {

   "source":"cat",

   "dest":"text"},

   {

   "source":"content",

   "dest":"text"},

   
   

   {

   "source":"author", "dest":"author_s"}]}}

   Get the entire schema in XML.

   
   

   
   

   curl http://localhost:8983/solr/gettingstarted/schema?wt=xml

   
   

   
   

   <response>

   <lst name="responseHeader">

   <int name="status">0</int>

   <int name="QTime">5</int>

   </lst>

   <lst name="schema">

   <str name="name">example</str>

   <float name="version">1.5</float>

   <str name="uniqueKey">id</str>

   <arr name="fieldTypes">

   <lst>

   <str name="name">alphaOnlySort</str>

   <str name="class">solr.TextField</str>

   <bool name="sortMissingLast">true</bool>

   <bool name="omitNorms">true</bool>

   <lst name="analyzer">

   <lst name="tokenizer">

   <str name="class">solr.KeywordTokenizerFactory</str>

   </lst>

   <arr name="filters">

   <lst>

   <str name="class">solr.LowerCaseFilterFactory</str>

   </lst>

   <lst>

   <str name="class">solr.TrimFilterFactory</str>

   </lst>

   <lst>

   <str name="class">solr.PatternReplaceFilterFactory</str>

   <str name="replace">all</str>

   <str name="replacement"/>

   <str name="pattern">([^a-z])</str>

   </lst>

   </arr>

   </lst>

   </lst>

   ...

   <lst>

   <str name="source">author</str>

   <str name="dest">author_s</str>

   </lst>

   </arr>

   </lst>

   </response>

   
   

   Get the entire schema in "schema.xml" format.

   
   

   
   

   curl http://localhost:8983/solr/gettingstarted/schema?wt=schema.xml

   
   

   
   

   <schema name="example" version="1.5">

   <uniqueKey>id</uniqueKey>

   <types>

   <fieldType name="alphaOnlySort" class="solr.TextField" sortMissingLast="true" omitNorms="true">

   <analyzer>

   <tokenizer class="solr.KeywordTokenizerFactory"/>

   <filter class="solr.LowerCaseFilterFactory"/>

   <filter class="solr.TrimFilterFactory"/>

   <filter class="solr.PatternReplaceFilterFactory" replace="all" replacement="" pattern="([^a-z])"/>

   </analyzer>

   </fieldType>

   ...

   <copyField source="url" dest="text"/>

   <copyField source="price" dest="price_c"/>

   <copyField source="author" dest="author_s"/>

   </schema>

   
   

   List Fields

   
   

   GET /collection/schema/fields

   GET /collection/schema/fields/fieldname

   
   

   INPUT

   Path Parameters

   
   

   Key	Description
   collection	The collection (or core) name.
   fieldname	The specific fieldname (if limiting request to a single field).

   
   

   Query Parameters

   The query parameters can be added to the API request after a '?'.

   
   

   Key	Type	Required	Default	Description
   wt	string	No	json	Defines the format of the response. The options are json or xml. If not specified, JSON will be returned by default.
   fl	string	No	(all fields)	Comma- or space-separated list of one or more fields to return. If not specified, all fields will be returned by default.
   includeDynamic	boolean	No	false	If true, and if the fl query parameter is specified or the field name path parameter is used, matching dynamic fields are included in the response and identified with the dynamicBa se property. If neither the fl query parameter nor the fieldn ame path parameter is specified, the includeDynamic quer y parameter is ignored. If false, matching dynamic fields will not be returned.

   
   

   

   
   

   showDefaults

   boolean

   No

   false

   If true, all default field properties from each field's field type will be included in the response (e.g. tokenized for solr. TextField). If false, only explicitly specified field properties will be included.

   
   

   OUTPUT

   Output Content

   The output will include each field and any defined configuration for each field. The defined configuration can vary for each field, but will minimally include the field name, the type, if it is indexed and if it is stored. If multiVa lued is defined as either true or false (most likely true), that will also be shown. See the section Defining Fields f or more information about each parameter.

   
   

   EXAMPLES

   Get a list of all fields.

   
   

   
   

   curl http://localhost:8983/solr/gettingstarted/schema/fields?wt=json

   
   

   The sample output below has been truncated to only show a few fields.

   
   

   
   

   {

   "fields": [

   {

   "indexed": true, "name": "_version_", "stored": true, "type": "long"

   },

   {

   "indexed": true, "name": "author", "stored": true, "type": "text_general"

   },

   {

   "indexed": true, "multiValued": true, "name": "cat", "stored": true, "type": "string"

   },

   ...

   ],

   "responseHeader": { "QTime": 1,

   "status": 0

   }

   }

   
   

   List Dynamic Fields

   
   

   GET /collection/schema/dynamicfields

   GET /collection/schema/dynamicfields/name

   INPUT

   Path Parameters

   
   

   Key	Description
   collection	The collection (or core) name.
   name	The name of the dynamic field rule (if limiting request to a single dynamic field rule).

   
   

   Query Parameters

   The query parameters can be added to the API request after a '?'.

   
   

   Key	Type	Required	Default	Description
   wt	string	No	json	Defines the format of the response. The options are json, xml . If not specified, JSON will be returned by default.
   showDefaults	boolean	No	false	If true, all default field properties from each dynamic field's field type will be included in the response (e.g. tokenized for solr.TextField). If false, only explicitly specified field properties will be included.

   
   

   OUTPUT

   Output Content

   The output will include each dynamic field rule and the defined configuration for each rule. The defined configuration can vary for each rule, but will minimally include the dynamic field name, the type, if it is indexed and if it is stored. See the section Dynamic Fields for more information about each parameter.

   
   

   EXAMPLES

   Get a list of all dynamic field declarations:

   
   

   
   

   curl http://localhost:8983/solr/gettingstarted/schema/dynamicfields?wt=json

   
   

   The sample output below has been truncated.

   
   

   
   

   {

   "dynamicFields": [

   {

   "indexed": true, "name": "*_coordinate", "stored": false, "type": "tdouble"

   },

   {

   "multiValued": true, "name": "ignored_*",

   "type": "ignored"

   },

   {

   "name": "random_*",

   "type": "random"

   },

   {

   "indexed": true, "multiValued": true, "name": "attr_*", "stored": true, "type": "text_general"

   },

   {

   "indexed": true, "multiValued": true, "name": "*_txt", "stored": true, "type": "text_general"

   }

   ...

   ],

   "responseHeader": { "QTime": 1,

   "status": 0

   }

   }

   
   

   List Field Types

   
   

   GET /collection/schema/fieldtypes

   GET /collection/schema/fieldtypes/name

   
   

   INPUT

   Path Parameters

   
   

   Key	Description
   collection	The collection (or core) name.
   name	The name of the field type (if limiting request to a single field type).

   
   

   Query Parameters

   The query parameters can be added to the API request after a '?'.

   
   

   Key	Type	Required	Default	Description
   wt	string	No	json	Defines the format of the response. The options are json or x ml. If not specified, JSON will be returned by default.
   showDefaults	boolean	No	false	If true, all default field properties from each field type will be included in the response (e.g. tokenized for solr.TextField). If false, only explicitly specified field properties will be included.

   
   

   OUTPUT

   Output Content

   The output will include each field type and any defined configuration for the type. The defined configuration can vary for each type, but will minimally include the field type name and the class. If query or index analyzers, tokenizers, or filters are defined, those will also be shown with other defined parameters. See the section Solr Field Types for more information about how to configure various types of fields.

   
   

   EXAMPLES

   Get a list of all field types.

   
   

   
   

   curl http://localhost:8983/solr/gettingstarted/schema/fieldtypes?wt=json

   
   

   The sample output below has been truncated to show a few different field types from different parts of the list.

   
   

   
   

   {

   "fieldTypes": [

   {

   "analyzer": {

   "class": "solr.TokenizerChain", "filters": [

   {

   "class": "solr.LowerCaseFilterFactory"

   },

   {

   "class": "solr.TrimFilterFactory"

   },

   {

   "class": "solr.PatternReplaceFilterFactory", "pattern": "([^a-z])",

   "replace": "all",

   "replacement": ""

   }

   ],

   "tokenizer": {

   "class": "solr.KeywordTokenizerFactory"

   }

   },

   "class": "solr.TextField", "dynamicFields": [],

   "fields": [],

   "name": "alphaOnlySort", "omitNorms": true, "sortMissingLast": true

   },

   ...

   {

   "class": "solr.TrieFloatField", "dynamicFields": [

   "*_fs", "*_f"

   ],

   "fields": [

   "price", "weight"

   ],

   "name": "float", "positionIncrementGap": "0",

   "precisionStep": "0"

   },

   ...

   }

   
   

   List Copy Fields

   
   

   GET /collection/schema/copyfields

   
   

   INPUT

   Path Parameters

   
   

   

   
   

   Key	Description
   collection	The collection (or core) name.

   
   

   Query Parameters

   The query parameters can be added to the API request after a '?'.

   
   

   Key	Type	Required	Default	Description
   wt	string	No	json	Defines the format of the response. The options are json or xml. If not specified, JSON will be returned by default.
   source.fl	string	No	(all source fields)	Comma- or space-separated list of one or more copyField source fields to include in the response - copyField directives with all other source fields will be excluded from the response. If not specified, all copyField-s will be included in the response.
   dest.fl	string	No	(all dest fields)	Comma- or space-separated list of one or more copyField dest fields to include in the response - copyField directives with all other dest fields will be excluded. If not specified, all copyField-s will be included in the response.

   
   

   OUTPUT

   Output Content

   The output will include the source and destination of each copy field rule defined in schema.xml. For more information about copying fields, see the section Copying Fields.

   
   

   EXAMPLES

   Get a list of all copyfields.

   
   

   
   

   curl http://localhost:8983/solr/gettingstarted/schema/copyfields?wt=json

   
   

   The sample output below has been truncated to the first few copy definitions.

   
   

   
   

   {

   "copyFields": [

   {

   "dest": "text", "source": "author"

   },

   {

   "dest": "text",

   "source": "cat"

   },

   {

   "dest": "text", "source": "content"

   },

   {

   "dest": "text", "source": "content_type"

   },

   ...

   ],

   "responseHeader": { "QTime": 3,

   "status": 0

   }

   }

   
   

   Show Schema Name

   
   

   GET /collection/schema/name

   
   

   INPUT

   Path Parameters

   
   

   Key	Description
   collection	The collection (or core) name.

   
   

   Query Parameters

   The query parameters can be added to the API request after a '?'.

   
   

   Key	Type	Required	Default	Description
   wt	string	No	json	Defines the format of the response. The options are json or xml. If not specified, JSON will be returned by default.

   
   

   OUTPUT

   Output Content

   The output will be simply the name given to the schema.

   
   

   EXAMPLES

   Get the schema name.

   
   

   
   

   curl http://localhost:8983/solr/gettingstarted/schema/name?wt=json

   
   

   
   

   {

   "responseHeader":{ "status":0,

   "QTime":1},

   "name":"example"}

   
   

   Show the Schema Version

   
   

   GET /collection/schema/version

   
   

   INPUT

   Path Parameters

   
   

   Key	Description
   collection	The collection (or core) name.

   
   

   Query Parameters

   The query parameters can be added to the API request after a '?'.

   
   

   Key	Type	Required	Default	Description
   wt	string	No	json	Defines the format of the response. The options are json or xml. If not specified, JSON will be returned by default.

   
   

   OUTPUT

   Output Content

   The output will simply be the schema version in use.

   
   

   EXAMPLES

   Get the schema version

   
   

   
   

   curl http://localhost:8983/solr/gettingstarted/schema/version?wt=json

   
   

   
   

   {

   "responseHeader":{ "status":0,

   "QTime":2},

   "version":1.5}

   
   

   List UniqueKey

   
   

   GET /collection/schema/uniquekey

   INPUT

   Path Parameters

   
   

   Key	Description
   collection	The collection (or core) name.

   
   

   Query Parameters

   The query parameters can be added to the API request after a '?'.

   
   

   Key	Type	Required	Default	Description
   wt	string	No	json	Defines the format of the response. The options are json or xml. If not specified, JSON will be returned by default.

   
   

   OUTPUT

   Output Content

   The output will include simply the field name that is defined as the uniqueKey for the index.

   
   

   EXAMPLES

   List the uniqueKey.

   
   

   
   

   curl http://localhost:8983/solr/gettingstarted/schema/uniquekey?wt=json

   
   

   
   

   {

   "responseHeader":{ "status":0,

   "QTime":2},

   "uniqueKey":"id"}

   
   

   Show Global Similarity

   
   

   GET /collection/schema/similarity

   
   

   INPUT

   Path Parameters

   
   

   Key	Description
   collection	The collection (or core) name.

   
   

   Query Parameters

   The query parameters can be added to the API request after a '?'.

   
   

   Key	Type	Required	Default	Description
   wt	string	No	json	Defines the format of the response. The options are json or xml. If not specified, JSON will be returned by default.

   OUTPUT

   Output Content

   The output will include the class name of the global similarity defined (if any).

   
   

   EXAMPLES

   Get the similarity implementation.

   
   

   
   

   curl http://localhost:8983/solr/gettingstarted/schema/similarity?wt=json

   
   

   
   

   {

   "responseHeader":{ "status":0,

   "QTime":1},

   "similarity":{ "class":"org.apache.solr.search.similarities.DefaultSimilarityFactory"}}

   
   

   Get the Default Query Operator

   
   

   GET /collection/schema/solrqueryparser/defaultoperator

   
   

   INPUT

   Path Parameters

   
   

   Key	Description
   collection	The collection (or core) name.

   
   

   Query Parameters

   The query parameters can be added to the API request after a '?'.

   
   

   Key	Type	Required	Default	Description
   wt	string	No	json	Defines the format of the response. The options are json or xml. If not specified, JSON will be returned by default.

   
   

   OUTPUT

   Output Content

   The output will include simply the default operator if none is defined by the user.

   
   

   EXAMPLES

   Get the default operator.

   
   

   
   

   curl http://localhost:8983/solr/gettingstarted/schema/solrqueryparser/defaultoperator?wt= json

   
   

   
   

   {

   "responseHeader":{ "status":0,

   "QTime":2},

   "defaultOperator":"OR"}

   
   

   Manage Resource Data

   
   

   The Managed Resources REST API provides a mechanism for any Solr plugin to expose resources that should support CRUD (Create, Read, Update, Delete) operations. Depending on what Field Types and Analyzers are configured in your Schema, additional /schema/ REST API paths may exist. See the Managed Resources secti on for more information and examples.

   
   

   Putting the Pieces Together‌

   At the highest level, schema.xml is structured as follows. This example is not real XML, but it gives you an idea of the structure of the file.

   
   

   
   

   <schema>

   <types>

   <fields>

   <uniqueKey>

   <copyField>

   </schema>

   
   

   

   Obviously, most of the excitement is in types and fields, where the field types and the actual field definitions live. These are supplemented by copyFields. The uniqueKey must always be defined. In older Solr versions you would find defaultSearchField and solrQueryParser tags as well, but although these still work they are deprecated and discouraged, see Other Schema Elements.

   
   

   Types and fields are optional tags

   Note that the types and fields sections are optional, meaning you are free to mix field, dynamicF ield, copyField and fieldType definitions on the top level. This allows for a more logical grouping of related tags in your schema.

   
   

   Choosing Appropriate Numeric Types

   
   

   For general numeric needs, use TrieIntField, TrieLongField, TrieFloatField, and TrieDoubleFiel d with precisionStep="0".

   If you expect users to make frequent range queries on numeric types, use the default precisionStep (by not specifying it) or specify it as precisionStep="8" (which is the default). This offers faster speed for range queries at the expense of increasing index size.

   
   

   Working With Text

   Handling text properly will make your users happy by providing them with the best possible results for text searches.

   One technique is using a text field as a catch-all for keyword searching. Most users are not sophisticated about their searches and the most common search is likely to be a simple keyword search. You can use copyField to take a variety of fields and funnel them all into a single text field for keyword searches. In the schema.xml file for the "techproducts" example included with Solr, copyField declarations are used to dump the contents of ca t, name, manu, features, and includes into a single field, text. In addition, it could be a good idea to copy ID into text in case users wanted to search for a particular product by passing its product number to a keyword search.

   Another technique is using copyField to use the same field in different ways. Suppose you have a field that is a list of authors, like this:

   Schildt, Herbert; Wolpert, Lewis; Davies, P.

   For searching by author, you could tokenize the field, convert to lower case, and strip out punctuation:

   
   

   schildt / herbert / wolpert / lewis / davies / p

   For sorting, just use an untokenized field, converted to lower case, with punctuation stripped:

   
   

   schildt herbert wolpert lewis davies p

   Finally, for faceting, use the primary author only via a StrField:

   Schildt, Herbert

   
   

   Related Topics

   
   

   

   SchemaXML

   
   

   DocValues‌

   DocValues are a way of recording field values internally that is more efficient for some purposes, such as sorting and faceting, than traditional indexing.

   
   

   Why DocValues?

   
   

   The standard way that Solr builds the index is with an inverted index. This style builds a list of terms found in all the documents in the index and next to each term is a list of documents that the term appears in (as well as how many times the term appears in that document). This makes search very fast - since users search by terms, having a ready list of term-to-document values makes the query process faster.

   For other features that we now commonly associate with search, such as sorting, faceting, and highlighting, this approach is not very efficient. The faceting engine, for example, must look up each term that appears in each document that will make up the result set and pull the document IDs in order to build the facet list. In Solr, this is maintained in memory, and can be slow to load (depending on the number of documents, terms, etc.).

   In Lucene 4.0, a new approach was introduced. DocValue fields are now column-oriented fields with a document-to-value mapping built at index time. This approach promises to relieve some of the memory requirements of the fieldCache and make lookups for faceting, sorting, and grouping much faster.

   
   

   Enabling DocValues

   
   

   To use docValues, you only need to enable it for a field that you will use it with. As with all schema design, you need to define a field type and then define fields of that type with docValues enabled. All of these actions are

   done in schema.xml.

   Enabling a field for docValues only requires adding docValues="true" to the field (or field type) definition, as in this example from the schema.xml of Solr's sample_techproducts_configs config set:

   
   

   
   

   <field name="manu_exact" type="string" indexed="false" stored="false" docValues="true" />

   
   

   If you have already indexed data into your Solr index, you will need to completely re-index your content after changing your field definitions in schema.xml in order to successfully use docValues.

   
   

   

   DocValues are only available for specific field types. The types chosen determine the underlying Lucene docValue type that will be used. The available Solr field types are:

   

   StrField and UUIDField.

   

   

   If the field is single-valued (i.e., multi-valued is false), Lucene will use the SORTED type. If the field is multi-valued, Lucene will use the SORTED_SET type.

   

   Any Trie* numeric fields, date fields and EnumField.

   

   

   If the field is single-valued (i.e., multi-valued is false), Lucene will use the NUMERIC type. If the field is multi-valued, Lucene will use the SORTED_SET type.

   These Lucene types are related to how the values are sorted and stored.

   There are two implications of multi-valued DocValues being stored as SORTED_SET types that should be kept in mind when combined with /export (and, by extension, Streaming Expression-based functionality):

   1. Values are returned in sorted order rather than the original input order.

   2. If multiple, identical entries are in the field in a single document, only one will be returned for that document.